The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a free tier that enables developers to easily use our platform, others see it as an opportunity for exploitation. Even more unfortunate is that it has a negative effect on legitimate users. It’s a fine balance that has been highlighted by several users within this thread.
Spam filtering is not a perfect science, and we’re constantly tweaking things -- with our customers in mind. This issue should be limited to new applications where the trust signal might be a bit lower. Thus existing apps / customers shouldn’t be experiencing issues (which was also highlighted by a few within this thread). If this isn’t the case email me: cramsdale@google.com. For those asking, “hey, why am I being penalized for being a new customer?” See my previous comment about spam filtering not being a perfect science. Then email me.
Just a quick thank you for the free tier that App Engine offers. I use it to provide an educational app for my wife, who's a teacher, that she uses as part of her classes. It's nice to be able to provide that to her and her class, without hitting the bank (the school does that enough...). So, thank you.
Long time App Engine user here. If you're sending email as part of your app, I'd recommend using Mailgun (or similar services). You can set up DKIM, inbound routes, etc, and generally have more control over how your mail is processed. I never got the impression App Engine really wanted to be in the email delivery business, so if that's important to you, there are tools that are focused on that (and free up to a monthly quota). My $0.02.
Love Mailgun. You can configure it to just be a very easy mail provider or use the APIs to send mail and turn on all kinds of things like campaign monitoring, tags, click opens, etc.
The best part for me when using it for customers has been the bounce/click through rate tracking. When someone asks me, "how do I know they got it?" It's incredibly nice to point them to
the dashboard and show a less than 1% bounce rate (people putting in bad email is almost always the reason) with a log of every single email sent.
Most my clients get this service for free because their volume us low enough. They have quite a generous free tier.
+1 as well, after the changes to Mandrill recently, was looking for a new provider and can also recommend Mailgun (at least, for ease and configurability in set up and testing so far, not yet in production).
Yes, Mandrill should have kept the free quota after DKIM and SPF verification. Now, I will be trying out the competitors (Sendgrid and Mailgun) and if I find them at par with Mandrill then for paid projects, I might use them rather than Mandrill.
For our clients, the appeal of Mandrill is the Mailchimp Template Editor. I think this is where their focus is going and so if you don't need that, then definitely check out competitors.
Out of around 18 projects, in which we have used Mandrill, only in 1 we have used the template editor. (I think only about half went for paid accounts when the project went live.) Most of our need was simple and quick transactional messages (Registration, Forgot Password, etc.). The main reason we chose Mandrill was ease of set up over Amazon SES. (Even though we had AWS accounts for all projects.)
Anyways, it's up to Mandrill to choose who they want to serve. I really liked the service though. Best of Luck.
Thanks for the suggestion! I was having issues with SendGrid and had never heard of Mailgun. Took literally 2 minutes to get set up on Mailgun, add the credentials to my app, and send my first email! It's a super small app so don't need anything fancy, Mailgun is perfect.
edit: Not to say Mailgun isn't fancy, I have no idea if it is or not, I can say that it works in minutes though.
> This issue should be limited to new applications where the trust signal might be a bit lower. Thus existing apps / customers shouldn’t be experiencing issues.
1. My app is not new. It's been running without issue for 2 or 3 years.
2. My app is not on the free tier.
3. On average, my app sends out under 8 emails a day, and it's done the same for over 2 years.
How your algorithm considers my app a spam risk sure beats me.
I'm missing 11 days worth of quote requests from customers. Are these recoverable? (is there a hidden outgoing-spam bin?)
Possible reason that email service providers and other email sending services might see an increase in new signups and free tier use recently: http://blog.mandrill.com/important-changes-to-mandrill.html Mandrill in essence raised their price, eliminated free tier, and required prepurchased blocks of sending capacity IIRC
The numbers Mandrill released about that business suggest that they had a large number of low volume senders, who may now be looking for a new home.
As an email service provider it's like that and more, since (1) once an abuser uses your service, they've gotten the benefit immediately and keep it even if their account is discovered as fraudulent later, e.g. stolen CC number & chargeback. (2) Abusive users can directly harm good users such as by harming the deliverability of the overall platform. It's not just bad debt, it's bad experience too. (3) Unlike Candy Japan where fraudsters mostly just wanted to check CC numbers and not actually buy product, email abusers really want to send emails (4) It can be hard to tell good and bad senders apart because some companies with an internet presence aren't email savvy and might make mistakes or might get hacked.
Spam filters are always tough because if you give someone transparency into which actions of theirs that you consider abuse, then they will quickly detect and route around your attempt to block them. (See Candy Japan article) It's pretty easy for a human to guess what might be the sign of their fraud and run a few experiments to see what gets flagged e.g. By comparison a machine learning system might be hard to outsmart, but then it's also challenging to explain and troubleshoot false positives. Hence what's effective is often a combination of machine-learned filters and heuristics along with manual overrides by human judgment.
All other things equal, new users are a lot more likely to engage in fraud than existing ones, and so tend to be under more suspicion. Aside from B2B fraud where companies take out lines of credit and then go bankrupt intentionally, it's uncommon for existing established customers to turn fraudulent - they're already vetted. (Consider: who is more likely to be fraudulent. The first time subscriber to Candy Japan, or a subscriber who has been using it for 12 months and is about to buy their 13th month?) It's not a great experience as a new user to be under suspicion, but if it's temporary and easily overridden by a human it can be a decent trade-off - the need to reach out acts a deterrent to spammers but does not deter legitimate users as much (speaking generally).
Chris, you say Gmail tightened up their spam filters... Did this take place in the last ~6 weeks or so?
I've noticed the spam filter on Gmail for Google Apps has gotten significantly less accurate recently, resulting in far more false-positives than usual. Any ideas if this is a known issue? I can only presume it's more accurate overall, but it was definitely a noticeable change for our organisation.
The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a free tier that enables developers to easily use our platform, others see it as an opportunity for exploitation. Even more unfortunate is that it has a negative effect on legitimate users. It’s a fine balance that has been highlighted by several users within this thread.
Spam filtering is not a perfect science, and we’re constantly tweaking things -- with our customers in mind. This issue should be limited to new applications where the trust signal might be a bit lower. Thus existing apps / customers shouldn’t be experiencing issues (which was also highlighted by a few within this thread). If this isn’t the case email me: cramsdale@google.com. For those asking, “hey, why am I being penalized for being a new customer?” See my previous comment about spam filtering not being a perfect science. Then email me.
We’re here and we want to help.
-- Chris (Lead PM for App Engine)