Sending mail from your web server in parallel with google mail

I've been suffering for quite some time now with mail issues on the Philly CFUG website. Blog post announcements were being sent but marked as spam, so nobody was aware of meetings, so attendance took a nose dive. Big problem!

I've previously written that when you're using google mail for your domain, it can cause mail sent by your web server to look spammy. One approach to mitigating this problem is to send all outgoing mail through Google's SMTP servers. But of course there are restrictions for free accounts. You're limited to 500 outgoing messages per day, and there is a limit of recipients per message. This could be troublesome if you're sending out a high volume of password recovery mail, and things like that.

Sadly, even with Google SMTP configured for the CFUG blog, for whatever reason, mail wasn't going out. But there is another option...

When an email is received by the destination mail server, it (the server) checks the sender IP address against the domain's SPF record, and the SPF record that google instructs you to add includes their mail servers and marks everything else as "neutral":

v=spf1 include:_spf.google.com ~all

So what's so bad about this? For one, everyone and their grandmother's cat is using Gmail these days, and gmail takes that neutral ruling as "spammy enough for me!" and just marks the message as spam. How do I know that? Because GMail, like most mail servers, adds some debug information to the message headers. Here's the full text of a test message I sent myself using CFMail from my web server, before fixing my SPF record:

Received-SPF: neutral (google.com: 65.111.169.59 is neither permitted nor denied by best guess record for domain of noreply@phillycfug.org) client-ip=65.111.169.59;
Date: Fri, 16 Sep 2011 22:59:20 -0400 (EDT)
From: noreply@phillycfug.org
To: [manager]@phillycfug.org
Message-ID: <183870777.45.1316228360261.JavaMail.SYSTEM@127.0.0.1>
Subject: test
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

fingers crossed!

Take note of the line starting "Received-SPF:" -- this is added by GMail to document how spammy the message appears. The ultimate designation is "neutral", and then it proceeds to explain why. What we want it to say is "pass". The client-ip listed, 65.111.169.59, is the IP of the server sending the message; my web server in this case.

So how can we fix this? Fix the SPF record. I've looked into this a few times in the past, and it always seemed like black magic to me. Sure, there is lots of documentation available on the internet, but every time I looked at it, it looked worse than the worst man pages. I was never able to grok it.

Then last week I found OpenSPF, and their simple and clean explanation of SPF syntax.

It turns out that the fix is simple. Just change the above SPF record to the following:

v=spf1  a  mx include:_spf.google.com ~all

I've added the string "a mx" to the middle of the record. This indicates that in addition to the _spf.google.com record that allows all of google's (presumably numerous) mail servers, but also include every IP listed as an A record for the domain, and every IP from MX records. You may not need the MX part, but to be honest now that it's working I don't want to change it!

Sending another test message, the Received-SPF header line now reads:

Received-SPF: pass (google.com: domain of [manager]@phillycfug.org designates 65.111.169.59 as permitted sender) client-ip=65.111.169.59;

If my calculations are correct, group attendance is about to come back up. :)

in Google | Learning | 1 Response 2011-09-19 08:07

1 response:

Chris 01 Dec 2011 5:28 AM You are right! There are a lot of topics about getting your SPF records right, but after some searching, this was the first clear posting of what I needed. I have added this in.. Just have to wait and see if it solves my issue. Thanks for sticking this up!

fusiongrokker