Browse

Blog

More protection with Spamhaus HBL add-on

The Halon MTA provides the widest support for threat protection vendors. It effortlessly integrates with any threat intelligence feed or content filter engine thanks to its scripting language, and we have published several ready-made integrations in our module library. We also sell a bundled license that includes spam, phishing and malware protection from Cyren and Sophos in addition to the MTA platform subscription.

Our integrated package (available as VMs or cloud images) comes with several engines installed, and we just added Spamhaus DQS. It includes support for Spamhaus’ latest developments; the hash blocklist (HBL). It’s a form of anti-virus so that you can filter additional malicious content. Using the HBL in conjunction with IP and domain blocklists, you can secure catch rates of 99.71%* and false positives as low as 0%. You can trial it free for 30 days.

In short, the HBL is a list of cryptographic hashes used to block different elements of malicious email content e.g. a cryptowallet address. Each hash forms a unique string, creating a one-off identifier for that specific piece of content, without using any personally identifiable data. Spamhaus’ HBL includes listings of compromised email addresses, malware files, and crypto wallet addresses. The malware part of the HBL acts like a type of anti-virus protection for your email stream. Typically, to identify malicious email, Spamhaus would assess the sender’s IP and/ or domain reputation. When an email account is from an ESP, like Gmail or Outlook, listings aren’t always possible, otherwise multiple users would be blocked. However, using a hash means Spamhaus can associate malicious email emitting from a specific, compromised email address.

With pioneering collection and distribution methods, Spamhaus’ researchers and engineers hash malware files, listing them in under 20 seconds after detection. By querying the Spamhaus Malware HBL, the return codes will highlight:

  1. If the file is malicious. Here, the file has been assessed by Spamhaus Malware Labs and confirmed as known malware. The return code will also provide the family of malware. 
  2. Or, if the file is suspicious: it has been observed in spam, thereby making it suspicious, though its intent is not confirmed as malicious by Malware labs. Extreme caution should still be taken when handling this file.

Another piece of content used to catch nefarious activity, when IPs and domains can’t be associated with spam, are cryptowallets. Spammers often request payment in cryptocurrencies like Bitcoin and Dash, among others, when trying to extort money from victims. Spamhaus hashes these cryptowallet addresses.  This enables email administrators to easily block the well-known ‘sextortion’ emails. You know the ones…

“I’ve hacked your account. I’ve been watching you visit adult sites. I have made a recording of you from your webcam. Send me money via my Bitcoin address 1N6dubqFmnyQ2DWvi32ppVbc3kKMTYcGW or I’ll share the recording with your contacts”.

To trial the Spamhaus blocklists, including the HBL, sign up for the Spamhaus Data Query Service (DQS). After signing up for the 30-day free trial, you will receive a DQS key. Entering this into the Halon MTA interface will get you protection with IP, Domain, and Hash blocklists. If you are already subscribing to Spamhaus’ IP and domain data, the HBL is available to you free of charge; just tick the HBL box on your Halon interface, and the rest will be done for you!

*Accurate as of March 2021. See here for the latest independent statics by Virus Bulletin.

Halon 5.6 with native plugins and Linux admin

Spring is approaching, and we’re getting ready to ship the next quarterly Halon MTA release, 5.6 “pingy”. For the first time, we’re shipping the web administration interface as separate Linux packages, making it more convenient to setup Halon clusters on Ubuntu and CentOS. This is great news for example if you’re using containers to build an elastic cluster for cost or scalability reasons.

The biggest news however is the new native plugin system, which complements Halon script and FFI. It allows you to add functions to the MTA written in any language compatible with the C library ABI (such as C++ and Go). In addition to adding your own HSL functions, it also enables you to work with the queue and delivery subsystems. Queue plugins can implement things like specific delivery times and customised IP warmup. Delivery plugins can be used to implement for example HTTP delivery to an object storage. Modules should employ the suspend/schedule pattern to avoid blocking, since the Halon MTA is event-based (uses asynchronous IO). This will be described in great detail in an upcoming blog post.

To give you an idea of how it works, consider the example below. It implements a simple sleep function in Go. The Halon_hsl_register function registers a sleep function in HSL. The sleep function itself dispatches a sleepTask, and immediately returns (suspends the script execution). Finally, sleepTask first sleeps, and then wakes the script up again (schedule).

// go build -buildmode c-shared -o sleep.so sleep.go
package main

// #cgo CFLAGS: -I/opt/halon/include
// #cgo LDFLAGS: -Wl,--unresolved-symbols=ignore-all
// #include <HalonMTA.h>
import "C"
import "unsafe"
import "time"

func main() {}
func Halon_version() C.int {
    return C.HALONMTA_PLUGIN_VERSION;
}
func sleepTask(hhc *C.HalonHSLContext, ret *C.HalonHSLValue, sec float64) {
    time.Sleep(time.Duration(sec * 1000.0) * time.Millisecond);
    C.HalonMTA_hsl_schedule(hhc);
}
func sleep(hhc *C.HalonHSLContext, args *C.HalonHSLArguments, ret *C.HalonHSLValue) {
    var t C.double = 0;
    var arg0 = C.HalonMTA_hsl_argument_list(args, 0);
    if (arg0 != nil)
        C.HalonMTA_hsl_value_get(arg0, C.HALONMTA_HSL_TYPE_NUMBER, unsafe.Pointer(&t), nil);
    go sleepTask(hhc, ret, float64(t));
    C.HalonMTA_hsl_suspend_return(ptr); // go can’t use HalonMTA_hsl_suspend()
}
func Halon_hsl_register(hhrc *C.HalonHSLRegisterContext) C.bool {
    C.HalonMTA_hsl_register_function(hhrc, C.CString("sleep"), nil); // go can’t easily get the C function point of an exported method. “Nil” will make the HalonMTA_hsl_register_function use dlsym()
    return true;
}

As usual, the release is packed with other various improvements such as;

  • The FFI now provides the email body (MIME) and other File classes as a C-compatible FILE natively, making it easier to access to the modified email body when integrating with C libraries.
  • The integrated (virtual machine) package now includes Spamhaus DQS which provides additional functionality such as HBL.
  • Support for CSV schemas when using import so that imported CSV files get correctly typed out of the box.
  • Added In-Reply-To header to DSNs to support threaded bounces, which makes it easier and quicker to know which message is associated with what bounce.

See the release notes and detailed changelog for a full list of changes. We hope that you will enjoy this new quarterly release! If you have any questions, or are new to Halon, don’t hesitate to reach out!

Not all successful DSN deploys are successful

Our latest release 5.5 adds support for the RFC 3461 DSN extension, and published a guide on how to get started. At first glance it may seem like a very nifty feature, and one may wonder why it’s not just enabled by default. Fact it, the DNS extension comes with a few peculiarities which need to be considered.

In many email environments, the receiver has multiple store-and-forward MTAs before the message reaches the recipient’s inbox. Each MTA in the path has a different purpose, and may reject the message (if it is spam, for example) rather than deliver it forward. This is what requires MTAs to send “failed” DSN notifications; when an MTA which isn’t your first (edge) rejects a message. The sender always has to be notified as per the transaction safety requirement of email.

One of the features provided by the DSN extension is that you may also request “delivered” notifications by the last MTA before the recipient’s inbox (or “relayed” notifications by the last MTA supporting the DSN extension). This lets the sender know when and if the message was delivered.

Thinking about it, this feature may be nice to have enabled in your MTA for outbound email. It would let you or your users know if and when a message is delivered; rather than only notifying you when and if delivery fails.

If you implement this feature for inbound traffic however, it would be one of your own MTA that would have to send those notifications out to anyone on the Internet that requested them. The notifications would go out from the last MTA in your inbound path. When messages are processed and routed internally, additional information that should not be visible to the sender are sometimes added. Allowing the sender to request a notification possibly containing some of this possibly sensitive information might not be ideal. This concern has be raised many times before, for example in the Exim documentation

Note: Supplying success-DSN messages has been criticised on privacy grounds; it can leak details of internal forwarding.

During our implementation and testing we enabled success notifications for some of our outbound traffic, to see which mailbox providers send such notifications. We were quite surprised to learn that Microsoft includes quite a lot of information regarding the recipients in their success notifications. We instantly reported this to the MSRC team but it was rejected as not being an issue 🙈

Amongst those hundred of lines of X-MS- headers you receive in their success notifications, I’m not sure if the last login time of the recipient is necessary. This is in fact the actual time when the recipient (eg. [email protected]) last checked their email:

X-MS-UserLastLogonTime: 10/1/2020 12:06:31 AM

Bottom line is; should you choose to implement this feature, make sure you do not add sensitive or unnecessary information to your success notifications that may expose your network internals or your customers’ personal information.

Halon 5.5 with DSN extension support

Winter is here, and so is our next quarterly Halon MTA release: 5.5 codename “mappy”. It’s slang for happy on a Monday, which is quite fitting! It comes packed with new features, functions and improvements which opens up even greater possibilities for building differentiated, efficient services.

Our large-scale senders will appreciate additional tools for IP warmup, ensuring deliverability by complying with receiver guidelines. You can now choose the order by which source IPs in a pool are chosen, as opposed to load balancing between all IPs. It allows you to use one IP primarily until it exceeds the threshold for daily warmup for a certain destination. Another feature which is especially useful for senders looking at migrating their email infrastructure to cloud environments (such as AWS) is outbound PROXY support. It enables you to use the powerful HAProxy load balancer rather than NAT, when sharing IPs between MTA instances.

Do you want to leverage the DSN extension to better track email delivery? If so, you’ll appreciate our fully-scriptable RFC 3461 implementation. It allows you to quickly and easily tailor the DSN behaviour exactly to your liking.

The Halon scripting language has been extended with new data storage classes, generator functions with yield, and several new functions for encoding. The strongly-typed Map() and Set() storage classes are performance-optimised alternatives to the more generic array type. Take our revamped DMARC implementation‘s public suffix list lookup for example. It parses a data file, then storing and caching the results in a Set() for efficient lookup of the slightly complex rules:

$rules = memory_fetch("public_suffix_list.dat", function ($k) {
		$rules = Set("string");
		$file = File("file://public_suffix_list.dat");
		if (!$file)
			return false;
		while (is_string($line = $file->readline()))
		{
			$line = str_strip($line);
			if ($line[0] == "/" or $line == "")
				continue;
			$rules->add($line);
		}
		memory_store("public_suffix_list.dat", $rules);
			return $rules;
	});

The integrated (VM) package has received several requested features. First of all, the built-in web server which hosts the web administration and JSON API can now be used to also host custom PHP scripts. It allows you to create additional API endpoints, which can write files used by the Halon script during email processing. One possible use case for this is to receive requests from legal interception systems. Secondly, it is now possible to add local DNS records to the built-in Unbound DNS resolver from the web administration. It can be useful when split-horizon DNS is unavailable, and you need to override some specific DNS record. Finally, it allows you to sort the built-in history on both finished and received time. As usual, the package is based on the latest version of FreeBSD (12.2, which was released a few weeks ago) and comes with updated third-party components.

We hope that you will enjoy this new quarterly release. Please see the release notes for more information, and don’t hesitate to reach out if you have any questions!

Would you like to “like” an email?

Social media and instant messaging (such as Slack) has made reactions like “thumbs up” 👍 and other emojis popular additions to everyday communication. Wouldn’t it be nice if you could quickly press like to confirm or endorse something proposed in an email, rather than having to write a reply?

Dave Crocker, well known for his work on email standards and a senior advisor for M3AAWG, recently submitted the draft “React: Indicating Summary Reaction to a Message”. It introduces email to the world of reaction emojis using an additional email header. It could be implemented in the email clients or webmail (MUAs), and doesn’t require any changes to the transport layer. It’s however an early draft with several considerations yet to be discussed, so I wouldn’t hold my breath just yet.

So, what could it look like? I don’t know, but it didn’t stop me from creating the mockup below!

Halon appoints Anders Långsved new CEO

Halon, the Swedish provider of email security and management solutions, has appointed Anders Långsved as new CEO. With a long background in leading roles in several IT startups and large companies, such as Argogroup (acquired by Ascom), Tele2, Irisity and Wrebit, Anders is now chartered to take Halon to the next levels, on a internationalised market.

Halon now enters a very exciting phase for further expansion where solid product development over the last couple of years has laid a foundation for international success. Anders now has the task to build a team that will enable Halon to realise this potential. Also, in the current environment, we see professional, secure and predictable email becoming even more important than ever before

Richard Berkling, Chairman

Knowing this team for some time now, as a previous board member, it’s great to now be able to work with them every day. Halon has a superb product offering and we are now ready to grow internationally!

Anders Långsved, CEO

Halon offers its secure Mail Transfer Agent to large email providers and senders, world wide. Based in Gothenburg with local development, sales takes to customers such as KPN, One.com and Telia Cygate.

Halon 5.4 with tailored bounces

Image by Dean Hochman

Sometimes you want to customise the bounce generation. It could be anything from translating bounces into another language or making them more user-friendly, to implementing standards such as SRS. In order to do so, you’ve had to override Halon’s default bounce generation. In our new release Halon 5.4 “bouncy”, we’ve added many options to the built-in bounce generator. This enables you to quickly tailor the way bounces look and work, without having to reimplement the default generator.

Queue([
  "delay" => 3600,
  "dsn_delayed" => true,
  "dsn" => [
    "readable_mimepart" => MIME()
      ->setType("text/html")
      ->setBody("<em>choo choo train is delayed</em>")
      ->toString(),
    "original_headers" => false,
    "subject_prepend" => "Not making it in time: ",
    "headers" => [
      "Foo: Bar"
    ]
  ]
]);

As usual, the release comes with many other improvements. The foreign function interface (FFI), which is used for loading external libraries into the MTA executable, can now export File classes as a C++ std::istream and X.509 resources as OpenSSL pointers. This powerful feature enables you to access the (potentially modified) email body as a virtual file. We’ve also added chunking (BDAT) for both sending and receiving, a zero-fill right shift bitwise operator, a data callback to the http() function, FFI function callbacks, and much more. Please see the release notes for a complete list of changes. We hope that you will enjoy this release. If you are new to Halon, don’t hesitate to contact us if you have any questions.

Why concurrency is important for large senders

Photo by Giuseppe Milo

Senders of large email volumes rely on the concept of multiple virtual queues for efficient delivery; separating email from different customers, to different destinations. This guarantees that for example one customer with a gigantic queue, or a destination which is tarpitting, doesn’t affect delivery of other traffic. Email transactions can take some time to complete, often as a result of content filtering at the destination. It is therefore necessary to support many simultaneous (concurrent) connections in each virtual queue, to support the throughput required for timely delivery. If each message takes an average of 1 second to deliver, 50 concurrent connections are needed to deliver 50 email per second.

The total maximum concurrency needed is the number of virtual queues multiplied with the desired concurrency for each queue. Traditional MTAs uses a process- or thread-based connection model. Those models scales poorly, and consequentially struggles to support more than a few thousand concurrent connections per server. This limitation becomes a problem as the system is scaled up, leading to bottlenecks or low customer density per MTA instance.

The Halon MTA uses a modern event-based connection model, sometimes called asynchronous or non-blocking. It scales exceptionally well; supporting tens of thousands of concurrent connections. Multiple worker threads are used to leverage all available CPU cores. This eliminates bottlenecks and can reduce the number of instances needed, which translates to lowered costs and simplified management.

Being asynchronous have benefits for receiving and processing inbound traffic as well. Consider the scenario where thousands of slow, or even idle, clients are connected to an MTA. Traditional MTAs would have to defer new connections from potentially legitimate senders, disturbing the service. Asynchronous MTAs like Halon are on the other hand be able to cope with a significantly large number of concurrent connections, and would handle the scenario above without breaking a sweat.

If you’re interested in learning more, don’t hesitate to reach out.

Halon 5.3 with new powerful queue API

Soap drop by Breic

Since the first Halon MTA release in 2008, we’ve had a text-based queue query syntax called HQL (a play on SQL) as part of our SOAP API. While it has served us well during all those years, it was time to move on to something more modern. The new 5.3 release (codename “buffy”) comes with a Protocol Buffers and JSON API which introduces a programmatic approach to queue operations.

The request and response body schemas are available on our Github page. For your convenience, the QueueList, QueueGroupByQueueUnload and QueueUpdate requests all have the same Condition argument. Those API calls can operate on both the active and defer queues, as well as messages on hold. That is why the condition argument both contains things like retry count, as well as resolved remote MX and IP. You can specify as many conditions as you like, and create logic-or expressions by specifying multiple conditions of the same type. There are exact matching, regular expressions, and intervals for integers and date/time. Needless to say; incredibly powerful.

The QueueGroupBy call returns the distribution based on the grouping parameters and intervals you choose; such as number of messages in various age buckets, grouped by recipient domain. This is useful for getting an overview of a large queue. Queries are blazingly fast, even with very large queues. All queue metadata (essentially the fields available as conditions) is loaded into memory, in order for the virtual sub-queues to work.

Halon 5.3 also comes with a new CLI called halonctl. It happens to be very useful when working with our API, as it can output the API request and response bodies for the command you run in JSON format. As you can se in the example below, the request body is printed first: 

$ halonctl queue update --bounce --state DEFER --jobid foobar --json-request --json
{
    "conditions": {
        "queues": [
            {
                "queue": "DEFER"
                ...

The CLI covers all the functionality of the product, and is a great complement to the web administration. Its configuration management sub-commands are useful for integrating Halon MTA instances into provisioning, deployment and CI/CD toolchains such as Puppet or Chef, where running commands is easier than making API calls.

Halon 5.3 comes with many other great improvements; such as connection pooling, a more efficient queue quota function, a new on-disk queue format and an Iconv() class for internationalisation conversion. Please see the release notes for a complete list of changes. We hope that you will enjoy this release as much as we do! If you are new to Halon, don’t hesitate to contact us, or dig into all our documentation that is available publicly on our website.

Just as cool as building it from scratch – why an email platform is your new best friend

For those of you working with larger amounts of traffic, which solution is better; building an email platform from scratch, or using an email security and delivery platform (ESDP) service? In this blog post, we describe both solutions so you can determine which one suits your company best.

Read more.