Browse

Blog

Centralised and cluster-synchronised rate limiting

The next e-mail security (SP) release will feature an improved rate limit implementation. As before, it provides a high-performance, roughly O(log N + log M), and easy-to-use rate limiting. However, it is now managed by a new (privilege-separated) process called rated, making it possible to build inter-flow rate controls. It also provides a light-weight cluster synchronisation over UDP, effectively making all cluster nodes able to collaborate.

Our customers use our rate limiting for all kinds of tasks, such as preventing users and websites from sending too many suspicious messages per day. You can read more on our wiki’s rate limit page. One additional benefit of the centralised architecture, is that one can view and manipulate mail flows’ rate limits from the hsh; Halon scripting language shell, which is accessible using SSH or from the /scripting/ page.

As you can see in the image, the rate limit page now shows both local hits (that is has accumulated itself) as well as the total number of hits, as synchronised by its peers.

Halon 3.1 based on FreeBSD 9.2 and restructured UI

We’re just about to release a new version (codename “lucky”) of our e-mail security gateway (SP), based on the latest FreeBSD version (9.2) in an attempt to track new FreeBSD versions as closely as possible. Our users will get benefits directly out of the amazing work of the FreeBSD team;

  • Improved performance in many areas, increasing the throughput of the system
  • Support for KVM’s VirtIO, allowing for more efficient network and disk accesses
  • Numerous updates in drivers, services, external projects, etc

In addition to that, we’ve improved the web interface even more, making it more convenient for customers with large clusters and traffic volumes. For example, the rate control page now has thresholds, and persistent search filters. Another new feature allows you to create interval statistics (for example “daily”) using the the rate() function instead of stat(), with a script such as

rate("daily-stat", "delivered", 10000000, 3600*24);

and use that when creating pie chart such as

delivered=r("daily-stat,delivered")
blocked=r("daily-stat,total")-r("daily-stat,delivered")

Finally, the web interface’s menu is simplified, and if you’re running a cluster, your default page will be a cluster overview.
blog-sp31
The logic behind the new menu, is to avoid having two “Activity” menus. Instead, both clustered and non-clustered versions of those pages (reporting, mail tracking, logging and rate control) are accessed from the same menu. The “node selector” in the upper right corner is used to switch between specific nodes, or the cluster as a whole. All configuration (which is mainly clustered, except for network settings and node overrides) has its own place in the menu, indicated with the dark blue color. Configuration management such as revision management and plain-text editing is placed in the bottom of the Configuration menu.

We hope you will enjoy this new release, and appreciate all feedback we can get!

Halon at CapTech Expo Oct 24th

You’ll find us in our booth (no 38) where you’ll not only be able to speak to our team members, but also get the opportunity to win our new HSR-603. Check-out our poster to find out more how to compete (poster in Swedish). Competing needs no specific language skills however…

Say what “new HSR-603”? Give us a ring (+46 31 301 19 20) or drop us a line ([email protected]) if you won’t be able to make it, and we’ll tell you all about it.

Sneak peek of Halon 3.0

We have some exciting news about our spam prevention series. The upgrade to FreeBSD 9 and overall refactoring was not the only treatment the SP series got this autumn and winter. We have collected feedback and performed evaluations of how our customers uses the web interfaces, trying to figure out what the best possible reporting and logging experience would be like. Read on to see what this has resulted in.

We have migrated to the new web interface from the security router series. That means a prettier UI, faster loading times, the ability to link directly to certain views using URLs with query strings, and better utilisation of your screen’s full width.

Let’s start with the mail tracking. The new UI provides some benefits of its own; displaying more information, auto-scaling all columns, and faster loading. We have combined the history, queue and quarantine within the same page. It’s pageable with a variable page size, so that you can view as many messages as you like per page. It has multi-select actions, for better queue management (viewing perhaps 1000 messages matching a certain search query, and bouncing them all). Finally, the “eye” icon brings up an inspector which you can use to view details for a message by just hovering items in the list.

The new log searcher is a lot faster than the previous, and can render thousands of lines without hogging your web browser. Most importantly, it can search multiple cluster nodes at the same time, viewing the number of hits (in real-time) per cluster node as a green badge. In that way, you can start a search for an IP address, and then ask someone to try sending the message again, and you will (when tailing in real-time) see a green badge on the cluster node which received the connection. Extremely handy.

The new reporting and graphs are based on the SR series code. That means a new statd which is fast, produces beautiful graphs, with real-time graphs, customisable legends, etc. Best of all is however that you can graph anything you like. To start with, you can create legends yourself; just look at the pie chart in the bottom right width the edit button clicked. You can even use math expressions to calculate values. Even cooler, you can use the new HSL stat() function in any flow, producing counters for whatever you like. There counters automatically becomes graphs and pie charts. I believe this is the most powerful reporting available in any mail security product ever. Perhaps any appliance.

Scripting, such as the system authentication script that allows for remote authentication and custom access levels, has become a lot better thanks to a great scripting editor with syntax highlighting and the ability to test the script using a “sandbox environment”.

The new web UI from the SR series doesn’t only bring nice real-time graphs, but also a true ANSI terminal.

We have made the already awesome clustering a lot easier to configure; with one “create cluster” guide joining two initial units, and one “add node” guide for adding a third, fourth, etc node to an existing cluster.

One spam accounting for ~80% of all traffic tonight

Have you received spam with subjects like

  • Tjana pengar pa ett socialt ansvarstagande arbete
  • Skapa ett battre liv for dina medmanniskor och tjana pengar pa det
  • Vi erbjuder dig ett arbete pa fritiden, lon fran 90 EUR i timman
  • Fa 90 EUR kontant i handen for den forsta timmens arbete inom tre dagar

you’re certainly not alone (and not using our spam filters). At about 7 pm yesterday (Swedish time) someone thought it would be a good idea to send a massive burst of spam. It seems that for many of our customers, that single spam outbreak accounted for as much as 70-90% of the total traffic. It seems that all of them used “yahoo.nl” as sender domain, which (unsurprisingly) doesn’t use SPF.

Fortunately, the combination of Commtouch’s RPD and our own (Halon) outbreak signatures was able to block it entirely, from 6 pm.

We can see that a lot of this was also blocked at IP level. The “normal” amount of IP blocks is almost invisible in the graphs, compared to the spam outbreak. I’ve removed the axis of the graphs, but let me tell you this. One of our customers, which is a large hosting provider, blocked more than 4 million of those per hour. That sure is a pretty persistent spammer.

Developing live graphs

We said to ourselves; “wouldn’t graphs that update every second with live data be useful”, and a few hours later the statd process was tweaked to output 1-second measurements of traffic, CPU, firewall states, etc. and the graph library was modified to dynamically populate data-points (in addition to the “historical” rrdtool file format support that it currently has).

API-wise, this translates into the commandRun API. The normal graphs, populated over time, is fetched using the graphFile API call, which takes an argument such as “interface-em0-packets” and returns the raw rrdtool database data. For real-time graphs, this translates into executing “statd -g interface-em0-packets” using the command-API. While we were at it, we added both “historical” and real-time graphs for firewall states.

In the web user interface, add graphs as usual, and select “Real-time” as time interval (instead of Recent, Day or whatever it says).

Halon 2.2.2.2 released with DNSSEC root trust

Today, on the 2nd of September we release 2.2.2.2. Neat, right?

Among the new features you’ll find the DNSSEC trusting the newly signed root anchor, administration user interface improvements and the usual stability and performance enhancements.

Now why would you care? Well, this could be your first step into the next generation of e-mail security. Why not start DKIM tagging when you’re at it?

There are small, yet useful features are well. Let’s say you want to implement a reporting rate control in your outgoing recipient flow, so that users or servers doesn’t send outgoing spam. In this example, we do this per-username ($saslusername, a pre-defined variable in the recipient flow) limiting the number of e-mail to 100 every hour, while sending at most one warning e-mail to the administrator about this every day and per user.

function myrate {
  if (rate("outbound", $saslusername, 100, 3600) == false) {
    $msg = "Rate "+$saslusername+" spam outbreak";
    if (rate("outbound-report", $saslusername, 1, 86400) == true) {
      mail("[email protected]", "[email protected]", "Rate", $msg);
    }
    Defer($msg);
  }
}

and then using it in code like

if ($saslauthed) {
  myrate();
  Accept();
}

Take care folks!

Halon attending PTS IPv6 seminar

Today we participated in a panel discussion about IPv6 deployment arranged by the Swedish Post and Telecom Agency (PTS). See the full video below (sorry, Swedish only).