DMARC: Microsoft, Facebook and Google unite to fight phishing - but will it work?

Filed Under: Featured, Google, Microsoft, Phishing, Spam

If the news wires are to be believed, the death of spam is imminent.

Again! (In 2004, Bill Gates famously remarked that spam would be a thing of the past by 2006.)

The prospective saviour this time round is a recently-proposed internet standard called DMARC, or Domain-based Message Authentication, Reporting & Conformance.

With Microsoft, Google, Facebook, PayPal, LinkedIn, Bank Of America and others behind it, this new kid on the anti-spam block has been reeling in coverage, with breathless headlines like this:

* Google, Microsoft Say DMARC Spec Stops Phishing

* Google, Facebook, Microsoft in PHISH-FIGHTING smackdown

* [DMARC] could dramatically slash the amount of spam received by hundreds of millions of people

If you're responsible for the mail infrastructure in your organisation, you might be a little sceptical at this point. You're probably asking yourself, "What happened to SPF and DKIM, which themselves were going to be the scourge of spammers?"

SPF stands for Sender Policy Framework. It allows an organisation to make a public declaration about which servers are authorised to send email on its behalf, thus - in theory - making phishing emails from imposters trivial to spot.

DKIM stands for DomainKeys Identified Mail. It allows an organisation to add a cryptographic signature to outbound email, thus - in theory - allowing an email to be validated against its sending domain.

The answer to your sceptical question about DMARC is that it doesn't replace SPF or DKIM, and it doesn't replace your current email security and control solution. In fact, it is predicated upon them, to the point that DMARC's official first step in its implementation guidelines is:

* Deploy DKIM & SPF. You have to cover the basics first.

And this begs the question, "How well are we doing with SPF so far?"

Let's take a look. I'll simplify greatly, but not, I hope, excessively.

SPF records are published using the Domain Name System (DNS). They declare which servers can, and which cannot, be considered official senders for a domain's email. Like this, for example.org:

v=spf1 +a:mx1.example.org +a:mx2.example.org -all

Interpreting this is easy. It means, "Email from the named servers mx1 and mx2 is genuinely ours. All other email claiming to be from us is bogus. Thanks for listening."

If I receive email purporting to come from example.org, I can unashamedly discard it unless it originates from one of the two named mail servers. In theory, it really is as easy as that.

The problem, in practice, is not that SPF has so far lacked a protocol layer on top of it to help you manage and tweak it usefully, which is most of what DMARC is about.

The problem is that few organisations have actually had the guts to use SPF to make a definitive statement about their email practices. They have paid SPF little more than lip-service.

To be of any real use, an SPF record must end with the word "all". This tells people what to do with email which doesn't come from one of your official servers. There are exactly four possibilities. With their official SPF name-codes, they are: +all (PASS), ?all (NEUTRAL), ~all (SOFTFAIL, somewhere between NEUTRAL and FAIL), and -all (FAIL, so block all offending emails).

Let me translate these SPF affirmations into my own name-codes, and into straightforward English:

+all DOH_EXCLAMATION Go on, spoof my domain! I don't really get this internet thing.
?all DONT_CARE I'm ambivalent about other people spoofing my domain, but I've read enough to know that +all looks silly.
~all PRETEND_TO_CARE Although I'm against phishing, I don't have the guts to be definitive. But I want to look better than the sort of person who would write ?all.
-all LISTEN_UP You have my official list of email servers. All other email is bogus. Thanks for listening.

Now let's take the fifteen DMARC contributors whose logos feature on the DMARC promotional page, and see where they stand on SPF. Remember, as DMARC says, you have to cover the basics first:

* agari.com: -all
* americangreetings.com: -all
* aol.com: ?all
* bankofamerica.com: ~all
* cloudmark.com: ~all
* comcast.com: no SPF record
* facebook.com: -all
* fidelity.com: -all
* google.com: ~all
* linkedin.com: ~all
* microsoft.com: ~all
* paypal.com: ~all
* returnpath.net: ~all
* trusteddomain.org: no SPF record
* yahoo.com: no SPF record

Doesn't bode too well, does it? Three no-shows, one DONT_CARE and seven PRETEND_TO_CAREs out of fifteen.

Does this mean DMARC is useless?

No. There are some good things in there.

Firstly, DMARC provides a means for recipients to report potentially out-of-policy emails to senders (whether they're blocked or not), thus helping senders gain confidence to move from PRETEND_TO_CARE to LISTEN_UP.

Secondly, DMARC provides a way to apply policy rules only in a random subset of cases, thus helping senders test the effectiveness of policy changes before making them live to the whole internet. (Your change control committee will love this.)

But will DMARC be the end of phishing and fraud in email?

I'll leave that as a rhetorical question.


-

, , , , , , , , , , ,

You might like

9 Responses to DMARC: Microsoft, Facebook and Google unite to fight phishing - but will it work?

  1. NiallG · 907 days ago

    If you look at an internet e-mail header - the Received part is normally inserted by sending and relay servers. if a message is relayed at some point the "passing" server may not be the source server (particularly if the server has to store and forward at some time) and even reverse DNS lookup can fail particularly if a corporate network uses an ISP and NAT and the ISP uses DHCP to assign addresses - consequently the relay servers IP address can change - so this idea only works for Static IP Addresses, so deploying it has the same problem as DNS reverse lookups. Also a Mail Server relaying to the internet does not have to have a DNS entry - only the mail server Receiving mails has to have an MX record.

    So such a system would only work for large and corporate Mail Systems - and even then may have problems for people who use their own mail client and not a web based mail system.

    What would be much more useful would be for the SMTP Protocol setup data to be used to generate the "From" and "To" fields displayed to users in their mail systems and not the internal "data" - as this can be faked easily, both the "From" and "to" fields...

    [Comment edited for length]

    Of course another issue with Phishing mail is that the text of the link looks like it points to a legitimate web site while the actual HTTP part is hidden from the user - so maybe a different way of displaying embedded links in messages would help e.g
    At present a link is shown as "mybank.com" with the destination hidden. Maybe a link should be displayed as follows "mybank.com(link:myfakewebsite.com)" would also ring alarm bells.

    Lets face it, it is the failure to display protocol and relay information and link destinations to mail recipients that allows spammers and phishers to craft mails that fool people and this is in the hands of the mail server and client and web providers and could be implemented immediately - exposing the fake mails to their recipients.

    I have worked in e-mail for 21 years and daily deal with virus, spam and phishing mail recipients and filters and have found that once mail recipients have explained to them the shortcomings of SMTP and how message transmission data is lost and link data is actually displayed the light dawns on them and they are empowered to beat these scams..

    I suspect that the major players are more interested in saving storage space and not protecting the recipients from mail scams which they leave to companies like Sophos and spam filtering software.

    • Paul Ducklin · 907 days ago

      My own opinion is that the reason so many large companies - ironically including plenty of financial institutions - aren't willing to "get serious" with SPF is because they are keen on using bulk mailouts, and willing to outsource this sort of marketing activity to third parties. So they can never actually be certain, at any moment, which servers might be sending email on their behalf.

      Ergo, if they were to publish a "-all" SPF record, any recipient who took them at their word might end up discarding email which was, in fact, legitimate - insofar as any advertising emails can be considered legitimate.

  2. MLM · 907 days ago

    I was just doing a test on the first domain in your list, agari.com and noticed that they inclide "_spf.google.com"
    Agari has -all in the spf record but _spf.google.com has ~all. How should this be interpreted on the receiving end?

    • Paul Ducklin · 906 days ago

      My understanding of RFC 4408 (SPF) is that "all" specifiers in includes do not terminate processing of the overall SPF result. So Agari's -all trumps Google's ~all. (This applies the other way around, too. An included -all is trumped by a less strict specification in the top-level record.)

      As the RFC admits, this makes the verb "include" a poor choice, since it doesn't literally work as if the text of the includee were inserted into the includer. The RFC suggests treating "include" as "if-pass". An included record can force the overall evaluation to pass, but not to PRETEND_TO_CARE or to LISTEN_UP.

  3. Ian Collier · 907 days ago

    SPF has a problem at any institution with personal email users (such as a university), because while the institution gives them an email address, they all prefer to use Gmail or Yahoo or Hotmail, or they send emails from home via their ISP's mail server. And of course they all put their institutional email address in the headers of these emails. So the institution can have no idea where legitimate email from these users might come from.

    SPF also won't work where a user leaves one email provider and has all their email forwarded to another, because now no matter who the forwarded email is from it will appear to come from the first email provider's mail server.

    (But I'm sure these cases are well discussed on the Internet anyway.)

  4. Nigel · 907 days ago

    I'm skeptical of any claim that it's possible to end spam with a single solution.

    For example, sender authentication via digital certificate would certainly reduce the amount of spam. Many (maybe even most) spammers don't want anyone to know who they are. If all incoming servers would only accept messages signed by senders with valid certs whose subject addresses matched the originating address, that would take a big bite out of the volume of spam traffic.

    But that wouldn't catch ALL the spammers, like the jerks in foreign nations who don't care whether you know who they are. And of course, as we've seen recently with bogus or stolen certs, even trusted certificate authorities (CAs) can turn out not to be so trustworthy.

    But I expect that there's a larger and more fundamental reason such a universal cert requirement won't happen. In order for it to work, ISPs and other mail service providers would have to require their users to use trusted certs, which would in turn require them to educate users about secure email practices in the first place. I suspect that there will be a blizzard in hell before that happens.

    The irony is that the ISPs and other mail service providers are the ones who would have the most to gain from implementing such measures. If the volume of spam were significantly reduced, so would be the costs of maintaining bandwidth and server infrastructure...not to mention the increased user experience quality for their customers. Apparently, enlightenment on the subject of mutual interest is a rare commodity in the industry.

  5. Roy · 906 days ago

    Does an SPF record at a third level domain (eg. xx.gov.au) affect any/all fourth level domains (eg. deet.xx.gov.au) in the absence of specific SPF records for those (200+) domains? Many of the sub-domains are owned by independent entities but have to be permitted in xx.gov.au because of the way gov.au was set up in the early nineties. I have seen the reverse where an ISP hosting a web server for abc,xx,gov,au added an SPF record for their own mail relay which prevented the central government mail relays from sending mail on behalf of abc.xx.gov.au. Unfortunately, since I retired from the job 4 years ago they have also implemented e-mail to/from xx.gov.au instead of from a subdomain of that.

    • Paul Ducklin · 905 days ago

      An SPF record does not automatically apply to subdomains.

      You can use a DNS wildcard record to achieve this outcome (e.g. by giving a list of approved mail servers for *.nsw.gov.au) but RFC 4408 advises you strongly not to.

      Given the importance placed by the federal government these days on protecting "critical infrastructure", I don't think it's unreasonable to expect every department to be able to account for the sources which are authorised to generate email on its behalf. (Try "dig ato.gov.au txt", for example, and you'll see an SPF record ending -all.)

  6. Robert Wurzburg · 906 days ago

    What ever happened to the Domain Keys project (RFC 4870?) Looks like it died on
    the vine.
    DKIM using Security Certificates, with MAC address validation and authentification
    would have been a good solution to stopping forged headers and phising emails.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

About the author

Paul Ducklin is a passionate security proselytiser. (That's like an evangelist, but more so!) He lives and breathes computer security, and would be happy for you to do so, too. Paul won the inaugural AusCERT Director's Award for Individual Excellence in Computer Security in 2009. Follow him on Twitter: @duckblog