Browse

Tag: spam

Halon and Spamhaus in email security partnership

We are excited to announce that Halon now provides official integration with Spamhaus Technology anti-spam & threat data feeds (IP & domain blocklists). Both companies worked together to ensure that this new functionality would be simple to deploy while also scaling all the way from smaller systems to large ISP’s with millions of users and complex email flows.

Read more.

What to remember when you hate on spam

Did you open your mailbox this morning, only to find a few more of those obnoxious spam emails? Did you react with an irritated frown and a thought about that you are paying for some service to keep this shit out? Don’t worry, it’s perfectly normal. In fact, so did I. Read more.

Why everybody should use DMARC to prevent phishing

Email is a major source of phishing and malware attacks. The Locky ransomware solely contributed to a 412% increase of malware emails in March compared to February, according to CYREN’s May 2016 cyberthreat report. While I believe that awareness and training is the most universally effective counter-measure, even that is really difficult, according to this recent study. We probably need a combination of training and technological advancements. One of the latter has to do with email authenticity. Can you trust an email’s sender adress? Generally no, but you can with DMARC. Read more.

Using reCAPTCHA to handle spam misclassification

Today’s leading spam filter technologies offer a very high degree of accuracy. In this blog I’ll describe the current state of spam classification, and propose a pretty innovative method that can significantly improve both senders’ and recipients’ satisfaction (as well as reducing the burden on administrators and support staff) by enabling senders to report false positives if they pass a CAPTCHA test. Let’s start by familiarising ourselves with the history of anti-spam.

Release blocked email

Background

The terminology that we normally use is

  • False positive, a blocked desired (legitimate) email (“ham”)
  • False negative, a missed spam that slipped through filters into a user’s mail box

Historically, spam filters had poor accuracy and low performance, and email was scanned after being accepted (probably as a consequence of the former). Finding themselves unable to reject email, they offered actions such as putting suspected spam in a junk folder, quarantine or by tagging the subject line.

This I believe, significantly damaged people’s trust in email as a reliable transport, simply because it makes legitimate (potentially important) email disappear.

The leading spam classification technologies today however, offers both high accuracy and performance. Many of them, including Cyren (that we use), uses fuzzy checksums (or “patterns”) to measure and classify email in a distributed, collaborative fashion. By constantly updating the hashing logic, anti-spam vendors are able to adopt as spammers evolve their tactics. By primarily looking at individual spam “outbreaks”, the false positive ratio is generally low in such systems. This is key, since people tend to be much less bothered by a few false negatives (missed spam) rather than having desired email blocked.

The high accuracy and performance also makes rejecting spam (rather than accepting it) a viable option. Rejecting spam is arguable superior to accepting and quarantining it, since the sender is informed about the email not being delivered to the recipient’s inbox. It reestablishes email as a reliable (transactionally safe) transport, while a copy of (the rejected) spam can still be retained in a quarantine of junk folder. Halon has advocated for this approach for a long time, and it’s a prerequisite for efficient feedback and reporting mechanisms like the one I’m going to describe now.

Using CAPTCHA to handle false positives

While I believe that our default approach of rejecting (giving a 500-error) spam with an informative error message (and storing a copy in a quarantine or junk folder) is superior to a traditional quarantine, there sure is room for improvement. For example, the sender needs to contact the recipient using some other mean (alternative email or phone, which they might not have), the quarantine might consume a significant amount of disk space, and the recipient might need to bother the support staff.

We’ve developed a self-service false-positive report and release project simply called sender-fp-release to address those shortcoming. As it says on its Github page, it allows senders to report false positives directly to the recipient after completing a reCAPTCHA.

sender-fp-release

In our experience, this system is a win for everybody;

  • The sender doesn’t need to manually contact the recipient, only verify a CAPTCHA
  • The recipient gets notified instantly, instead of having to browse through a junk folder
  • The helpdesk doesn’t need to do anything

Additionally, it saves disk space by only retaining spam for a short time (for example 1 day), unless the sender reports it. The retention time for reported email is extended (typically a week or two), giving the recipient plenty of time to release the email.

Release blocked email

If you believe that your spam handling could be improved, please take a look at the project, or maybe give it a spin.

Fight outbound spam and increase deliverability

Many email providers such as web hosts, ESPs and even VPS providers are familiar with the consequences of being blacklisted; angry customers calling the support because of delayed or reject email, countless of hours tracking down abusive users and patiently trying to get of the blacklists.

Unlike many other anti-spam products marketing themselves as “turn-key” solutions, Halon provides a scriptable email gateway that works as a toolbox for hosting providers. It enables them to tailor the system to fit them perfectly using our high-level scripting language. For example, you can in a programmable fashion create rate limits of anything you like. If you can identify customers based on their sender domain (enforced by the sending email server), you can defer messages based on the customer’s current deliverability statistics such as script such as

if (rate("delivery-failures", $senderdomain, 0, 3600) > 999)
    Defer("$senderdomain has more than 1000 failed deliveries during the last hour");
if (GetMailQueueMetric(["filter" => [ "senderdomain" => $senderdomain ]]) > 500)
    Defer("$senderdomain has exeeded the max queue limit of 500 messages");

Although quite different from inbound spam, filtering outbound spam can be extremely effective with the right tools, because you know who the sender is. In order to create a maintenance-free system, you can even allow a low rate of spam (per customer) sail through, to minimise the impact of false positives.

There are however many other factors that can be weighted into the equation. We have compiled a short list of the most common and effective methods to combat outbound spam which includes (but isn’t limited to);

Most of what we’ve discussed here works equally good in a fully transparent proxy installation, suitable for VPS providers that (for whatever reason) have chosen not to enforce the usage of an SMTP relay.

One spam accounting for ~80% of all traffic tonight

Have you received spam with subjects like

  • Tjana pengar pa ett socialt ansvarstagande arbete
  • Skapa ett battre liv for dina medmanniskor och tjana pengar pa det
  • Vi erbjuder dig ett arbete pa fritiden, lon fran 90 EUR i timman
  • Fa 90 EUR kontant i handen for den forsta timmens arbete inom tre dagar

you’re certainly not alone (and not using our spam filters). At about 7 pm yesterday (Swedish time) someone thought it would be a good idea to send a massive burst of spam. It seems that for many of our customers, that single spam outbreak accounted for as much as 70-90% of the total traffic. It seems that all of them used “yahoo.nl” as sender domain, which (unsurprisingly) doesn’t use SPF.

Fortunately, the combination of Commtouch’s RPD and our own (Halon) outbreak signatures was able to block it entirely, from 6 pm.

We can see that a lot of this was also blocked at IP level. The “normal” amount of IP blocks is almost invisible in the graphs, compared to the spam outbreak. I’ve removed the axis of the graphs, but let me tell you this. One of our customers, which is a large hosting provider, blocked more than 4 million of those per hour. That sure is a pretty persistent spammer.