We are excited to announce that Halon now provides official integration with Spamhaus Technology anti-spam & threat data feeds (IP & domain blocklists). Both companies worked together to ensure that this new functionality would be simple to deploy while also scaling all the way from smaller systems to large ISP’s with millions of users and complex email flows.
We asked Simon Forster of Spamhaus Technologies to describe what benefits he sees in this collaboration:
Spamhaus is looking forward to partnering with Halon to make email communications even safer for their clients. Coupled with Halon’s powerful scripting capabilities, it means clients can now prevent over 95% of spam and malware from getting into user’s mailboxes, without having to accept any data. Service providers can instantly recover the cost of bandwidth, servers & storage typically lost to accepting and processing spam.
The solution can also be used to block outbound spam which typically has links to fraudulent sites. Halon’s CTO Anders Berggren is equally excited:
We’re thrilled to collaborate with the Spamhaus Technology team. They are the most recognized name in IP & domain blocklists. This partnership furthers Halon’s mission to offer the highest performing and most comprehensive messaging platform. Halon enables service providers to build innovative, secure and very cost effective email solutions, and Spamhaus is a great addition to our platform.
Did you open your mailbox this morning, only to find a few more of those obnoxious spam emails? Did you react with an irritated frown and a thought about that you are paying for some service to keep this shit out? Don’t worry, it’s perfectly normal. In fact, so did I.
The fact of the matter is that however much we, or any of our colleagues in the email security business, claim that we do keep the spam out – nobody’s perfect. This is a constant game of hunting down the spammers and blocking them, and for each attack we stop, someone finds another way around and then we go after that. It never stops.
But the thing to consider when you see those two, or maybe even five spam emails, is that those are just the tip of the iceberg. In 2015 we sent about 205 billion emails per day, according to the Radicati Groups Email Statistics Report, and almost half of it was spam. The average business user protected by spam filters received 88 emails per day, out of which 12 were spam. That would mean an average catch rate of 86 %, while the anti-spam that Halon uses blocks significantly more.
So the spam you see is nowhere near the real amount of spam actually sent to you. And yes, sometimes a few of them break through the fences you have put up, and drives the industry to constantly improve. But we just want you to know that as long as you keep your hands off them spams, trash them without clicking any links, you are still safe. The war is ongoing, but we will keep you out of harms way. Get in touch if you need a hand!
Email is a major source of phishing and malware attacks. The Locky ransomware solely contributed to a 412% increase of malware emails in March compared to February, according to CYREN’s May 2016 cyberthreat report. While I believe that awareness and training is the most universally effective counter-measure, even that is really difficult, according to this recent study. We probably need a combination of training and technological advancements. One of the latter has to do with email authenticity. Can you trust an email’s sender adress? Generally no, but you can with DMARC.
To start with, an email has two sender addresses;
The “envelope” address, used as return address in case the email cannot be delivered. It’s communicated between MTAs in the MAIL FROM SMTP command.
The address written in the letter itself, that you see in your email app. It’s communicated as a so-called header named “From”.
Take this email for example, that our CFO Helena received a few months ago.
In Helena’s email app, it looks like an email coming from Peter, our CEO…
…when it in fact comes from someone else.
MAIL FROM:<[email protected]>
RCPT TO:<[email protected]>
Received: from p3plwbeout18-06.prod.phx3.secureserver.net (unknown [188.8.131.52])
by mail.halon.se (Halon) with ESMTPS
Mon, 23 May 2016 11:00:50 +0200 (CEST)
Received: from localhost ([184.108.40.206])
by p3plwbeout18-06.prod.phx3.secureserver.net with bizsmtp
id xl0p1s0013Hkz8V01l0p7m; Mon, 23 May 2016 02:00:49 -0700
Received: (qmail 23080 invoked by uid 99); 23 May 2016 09:00:49 -0000
Message-Id: <[email protected]mail18.godaddy.com>
From: "Peter Falck" <[email protected]>
Reply-To: "Peter Falck" <[email protected]>
To: [email protected] (cut)
The envelope address, in this case “channelforlove.com”, is protected by a thing called SPF. It’s an over 10 years old and widely deployed standard that prevents spammers and phishermen from spoofing the envelope address. SPF was a major achievement for email, because it enabled receivers (email administrators, spam filters, whitelists, etc) to trust the envelope’s sender domain.
The from address, in this case “halon.se”, has not been possible to verify, until DMARC came along. Because it’s the address that the user see in the email app, DMARC is a great tool for helping prevent scamming attacks.
This particular scam is pretty easy to see through, because of the poor language and the reply-to address. Other scam emails can obviously be quite effective. A recent study by a cybersecurity company suggests that ransomware is a billion-dollar business.
DMARC is already widely deployed by senders that would otherwise see their domains abused by scammers, such as PayPal and Facebook. The fact that it’s backed by large, influential enterprises and based on the existing standards SPF and DKIM, has probably been a contributing factor to its success. The specification was published in 2012, and within a year 60% of consumer mailboxes were protected. Support in software is quite good, and Halon implements it since 2013, as it’s part of our open source DKIM library.
Halon recommends that you start using DMARC, to help protect yourself and others from getting scammed. Contact us, and we’ll help your organisation set it up.
Today’s leading spam filter technologies offer a very high degree of accuracy. In this blog I’ll describe the current state of spam classification, and propose a pretty innovative method that can significantly improve both senders’ and recipients’ satisfaction (as well as reducing the burden on administrators and support staff) by enabling senders to report false positives if they pass a CAPTCHA test. Let’s start by familiarising ourselves with the history of anti-spam.
The terminology that we normally use is
False positive, a missed spam that slipped through filters into a user’s mail box
False negative, a blocked desired (legitimate) email (“ham”)
Historically, spam filters had poor accuracy and low performance, and email was scanned after being accepted (probably as a consequence of the former). Finding themselves unable to reject email, they offered actions such as putting suspected spam in a junk folder, quarantine or by tagging the subject line.
This I believe, significantly damaged people’s trust in email as a reliable transport, simply because it makes legitimate (potentially important) email disappear.
The leading spam classification technologies today however, offers both high accuracy and performance. Many of them, including Cyren (that we use), uses fuzzy checksums (or “patterns”) to measure and classify email in a distributed, collaborative fashion. By constantly updating the hashing logic, anti-spam vendors are able to adopt as spammers evolve their tactics. By primarily looking at individual spam “outbreaks”, the false positive ratio is generally low in such systems. This is key, since people tend to be much less bothered by a few false negatives (missed spam) rather than having desired email blocked.
The high accuracy and performance also makes rejecting spam (rather than accepting it) a viable option. Rejecting spam is arguable superior to accepting and quarantining it, since the sender is informed about the email not being delivered to the recipient’s inbox. It reestablishes email as a reliable (transactionally safe) transport, while a copy of (the rejected) spam can still be retained in a quarantine of junk folder. Halon has advocated for this approach for a long time, and it’s a prerequisite for efficient feedback and reporting mechanisms like the one I’m going to describe now.
Using CAPTCHA to handle false positives
While I believe that our default approach of rejecting (giving a 500-error) spam with an informative error message (and storing a copy in a quarantine or junk folder) is superior to a traditional quarantine, there sure is room for improvement. For example, the sender needs to contact the recipient using some other mean (alternative email or phone, which they might not have), the quarantine might consume a significant amount of disk space, and the recipient might need to bother the support staff.
We’ve developed a self-service false-positive report and release project simply called sender-fp-release to address those shortcoming. As it says on its Github page, it allows senders to report false positives directly to the recipient after completing a reCAPTCHA.
In our experience, this system is a win for everybody;
The sender doesn’t need to manually contact the recipient, only verify a CAPCHA
The recipient gets notified instantly, instead of having to browse through a junk folder
The helpdesk doesn’t need to do anything
Additionally, it saves disk space by only retaining spam for a short time (for example 1 day), unless the sender reports it. The retention time for reported email is extended (typically a week or two), giving the recipient plenty of time to release the email.
Many email providers such as web hosts, ESPs and even VPS providers are familiar with the consequences of being blacklisted; angry customers calling the support because of delayed or reject email, countless of hours tracking down abusive users and patiently trying to get of the blacklists.
Unlike many other anti-spam products marketing themselves as “turn-key” solutions, Halon provides a scriptable email gateway that works as a toolbox for hosting providers. It enables them to tailor the system to fit them perfectly using our high-level scripting language. For example, you can in a programmable fashion create rate limits of anything you like. If you can identify customers based on their sender domain (enforced by the sending email server), you can defer messages based on the customer’s current deliverability statistics such as script such as
if (rate("delivery-failures", $senderdomain, 0, 3600) > 999)
Defer("$senderdomain has more than 1000 failed deliveries during the last hour");
if (GetMailQueueMetric(["filter" => [ "senderdomain" => $senderdomain ]]) > 500)
Defer("$senderdomain has exeeded the max queue limit of 500 messages");
Although quite different from inbound spam, filtering outbound spam can be extremely effective with the right tools, because you know who the sender is. In order to create a maintenance-free system, you can even allow a low rate of spam (per customer) sail through, to minimise the impact of false positives.
There are however many other factors that can be weighted into the equation. We have compiled a short list of the most common and effective methods to combat outbound spam which includes (but isn’t limited to);
Using alternate IPs for suspected spam and source-hashing customers out through different IPs
Most of what we’ve discussed here works equally good in a fully transparent proxy installation, suitable for VPS providers that (for whatever reason) have chosen not to enforce the usage of an SMTP relay.
Halon is a flexible security and operations platform for in-transit email. It enables companies that build and operate large-scale
email services to offer competitive features by rapid implementation, and to lower costs of maintenance through
reliable deployment and reduced complexity.