How would I setup a "native" email system. I don't want to use some third party that I have to pay for. I'd like to be able to send the "raw" email myself.
So for example my current system uses Java + GMail's SMTP server. I have to pass it a username+password, and I can send basic emails that way. There are at least 2 major problems with this setup:
1) So far, all I have done is send REALLY basic 100% test based emails. It's not exactly clear to me how I would send a fully baked HTML email.
2) Consistently GMail BLOCKS my username. It requires me to manually go in at least every few days and enable "Allow less secure applications"
Basically you don't. It's a really big challenge to create a consistently non-blocked email sending service, especially if you host on a big cloud provider like AWS.
Sounds like you are using the Gmail API. To be honest that's not really fit for sending transactional emails, and is increasingly unfit for 3rd party consumption due to API restrictions.
Sendgrid also has a free tier that may work for your use case if the app is for a very limited audience, other than that you're kind of asking "how do I get something really difficult for free?"
Sendgrid uses a mail transfer agent like Postfix or PowerMTA (I don't actually know what they use but I wouldn't be surprised if it was PowerMTA) to talk to receiving email servers via SMTP.
It's not hard to set up the technical bits to send email. You basically just need to run an MTA that knows how to stamp your outgoing messages with DKIM (a cryptographic signature). Your server has to have an SSL certificate. You also need SPF records set up that accurately reflect what servers are allowed to send on your behalf. DMARC records are helpful as well but not required. If you're only sending as yourself, that's basically it. You should be able to get mail through to the big receivers without much trouble (maybe having to mark "not spam" a few times to start).
The trick to maintaining your sending reputation as an individual is to not send email for anyone else and never send content that you don't control. Sendgrid sends for other people and they have people on staff who's full time job is to fight with the big receivers to maintain their reputation.
Sendgrid puts a huge amount of money and resources into maintaining both technical and political legitimacy. They have relationships with other major senders and receivers of email, they follow every single rule to the T, they aggressively police their own IPs (of which they have many) etc. It's a complicated business, which is why it's totally worth it to pay them to do it for you.
Yes there is a real lesson about the pragmatic impossibility of true decentralization in email. Specialization of labor is a useful thing, it turns out.
That being said: the fact that is a protocol and not a proprietary standard does have decentralizing effects that do provide a useful incentive check against providers, even if there are only a few of them. There is also a real lesson there too, that one I'll leave for the reader.
How would I setup a "native" email system. I don't want to use some third party that I have to pay for. I'd like to be able to send the "raw" email myself.
So for example my current system uses Java + GMail's SMTP server. I have to pass it a username+password, and I can send basic emails that way. There are at least 2 major problems with this setup:
1) So far, all I have done is send REALLY basic 100% test based emails. It's not exactly clear to me how I would send a fully baked HTML email.
2) Consistently GMail BLOCKS my username. It requires me to manually go in at least every few days and enable "Allow less secure applications"