DMARC and You (And Your Web Applications)
DMARC stands for “Domain-based Message Authentication, Reporting and Conformance”. It has a nice ring to it; sounds very militarial. It’s a proposed IETF standard for changing the way that email messages are dealt with on the Internet.
The unofficial motto is “Rough Consensus and Running Code” – standards bodies have a reputation for being caught up in miles of red tape, and of over-designing specifications. Sometimes this has resulted in specifications that are not only difficult to implement, but impossible to implement wholly.
In this case, the rough consensus is happening between some of the larger email domains out there on the Internet. Google, Microsoft and Yahoo! handle between 1/3 and 2/3 of all email traffic out there – depending on sources – and they have always been at the front of the anti-spam and anti-phishing war.
As one of the Internet’s older standards, SMTP email (specified in RFC821) predates commercial use of the Internet, as well as widely available cryptography. For over 30 years, you have just been able to claim that you are allowed to send email for a given domain, without ever needing to (or being able to) prove that claim.
SPF originally stood for ‘Sender Permitted From’, but now stands for ‘Sender Policy Framework’. It allows a domain to specify who may send email on its behalf. Suddenly instead of the whole Internet being allowed to claim to send email from
@yahoo.com, only trusted IPs can.
An example SPF record looks like this:
insom.me.uk. IN TXT "v=spf1 mx -all"
This says that only the mail servers listed in the MX for this domain are allowed to send email, and email that fails this check should be dropped. That’s the
-all – if you just wanted to say that email that didn’t meet the SPF policy should be considered suspicious, then you could say
~all. If you didn’t want to demerit email that didn’t match you could say
Because SPF is often misconfigured, large email providers usually ignore the
-all option and treat those emails with suspicion, instead of outright rejection. This penalises those mail servers who have configured their policy correctly however, and is one factor leading to the introduction of DMARC.
DKIM is considerably more complicated than SPF, and uses cryptography to ensure messages are valid and from a permitted sender. Very loosely:
- The public part of a cryptographic key is published in the DNS record for a domain.
- The private part of the key is kept secret, and it is used to “sign” some fields in the email. If the email is fake or those fields are tampered with, then the signiture will not be verified.
- Recipients check the DKIM signature, if it’s available, and make their own decision as to merit or demerit an email based on the result.
Even moreso than SPF, DKIM does not specify what should happen if a signature matches, if it fails verification, or if no signature is present at all.
Once again, because of misconfiguration, its rare for email to be rejected outright by the larger email providers because of misconfigured DKIM. It simply increases the chance that an email will end up in someone’s spam folder and in some cases, doesn’t even preclude it from being delivered to the inbox.
This also penalises those providers with strict and accurate policies.
Back to DMARC
This is exactly where DMARC comes in.
Instead of replacing the previous standards, it augments them. You can publish a policy saying which of the above two standards you implement, and exactly what should happen if an email fails verification.
You can also specify an email address that should receive reports of successful or unsuccessful emails – usually batched into daily ZIP files. This helps solve the misconfiguration problem: email ISPs have a way to notify domains that emails are being rejected that may not otherwise be aware of.
These are the missing pieces to finally stopping the flow of fake emails from
@yahoo.com into everyone’s mail box: Yahoo! can now say “Only accept emails which come from the Yahoo! network, and which have a valid DKIM signature: reject everything else.”
Because there is a way to report undeliverable email back to Yahoo!, all other providers can confidently follow Yahoo!’s DMARC record, leaving invalid email totally undelivered.
In many, many ways, this is a giant move forward, and it’s great.
Why is bad, then?
It’s not bad by itself, but the old way of sending email has ruled for over thirty years. Lots of legitimate applications (not spam) will send email that claims to be from a domain that they don’t control. Many “send to a friend” features do this, as well as notification emails from support ticket systems and file transfer services. Mailing lists are also affected – any
sales@ address that includes people across several organisations will now face deliverability problems.
In my opinion, this inconvenience is worth it. Furthermore: it’s happening anyway, wheter we like it or not. This is the other truth about standards on the Internet: no one can force people to use or not to use them – and if the large email providers have decided that they’re adopting this standard (even if it never leaves the proposed draft stage that it is currently in) then that’s that.
For web developers, the new golden rule is:
Never send from a domain that you do not control.
That is, the envelope address and the
From: header should be your own. Something like
email@example.com will do. The text part of the header can still be the user, and the
Reply-To: header could and should be the user themselves.
For example, if Aaron wants to tell Neil about a great product, the email should look like:
Sender: firstname.lastname@example.org From: Aaron Brady via Great Products <email@example.com> To: Neil <firstname.lastname@example.org> Reply-To: <email@example.com> Subject: Check out this great product!
If your application isn’t doing this, you should update it: Yahoo! are the first large provider to publish a record recommending outright rejection of emails which don’t comply with their policy – but others are expected to start doing this over the next couple of years.
I wanted to link to this IETF email which was number #2 when I searched for DMARC – Re: Enough DMARC Whinging. There’s good points in there including in the quoted section, primarily that the owner of a top-level DNS name is really the one who gets to make the decision about how their email is handled – if you don’t like it, run your own domain or choose a new provider. This is harsh, but this harshness is explained away in this great line:
Are we engineers or curators of the past?
Like it or not, DMARC represents important progress in the future of email.
Want to discuss a project?
Talk to our Magento experts on 01785 279920