Browse

Author: Anders Berggren

Adding BIMI to your mailbox service to improve security and user experience

Image by Valimail

BIMI (Brand Indicators for Message Identification) is an emerging industry standard which attempts to drive adoption of strong email authentication, to help fight phishing, spoofing, and fraudulent emails by displaying verified brand images in the inbox. Several mailbox providers, including Gmail, AOL and Fastmail, are now live with BIMI. Are you looking to increase the security and experience for the users of your mailbox service by implementing this new standard as well? With Halon MTA, you’re ready to get started. Our BIMI module does the heavy lifting; verifying the indicators for inbound email and attaching the images to the email for display in the users’ inbox. Currently there are very few MUAs with BIMI support, but if you develop your own webmail or app the BIMI group has guidance for how to display the verified logo in your user interface.

What BIMI is, and how it works

In short, BIMI displays verified brand images in users’ inboxes, visible even before messages are opened. It builds upon the existing email authentication standards (SPF, DKIM and DMARC), and incentives email senders to embrace those technologies by increasing their brand impression. According to Agari, only 24% of Fortune 500 sending domains are authenticated. BIMI improves visibility and engagement for the brands, while at the same time increasing security for users. It roughly consists of the following steps;

  1. The brand sending the email first needs to implement email authentication (enforcing DMARC) for their domain name.
  2. The brand then creates an appropriate image of their trademarked logo, optionally applies for a Verified Mark Certificate (VMC), stores it on a HTTPS location, and points at it with a BIMI DNS record.
  3. When a BIMI-enabled mailbox service (like Gmail) receives an email from the brand to a specific recipient, the mailbox service’s MTA first verifies that the email is in fact from the sending brand using email authentication like DMARC.
  4. If the email is successfully authenticated, the mailbox service’s MTA checks if the sending brand’s domain has BIMI, and if so, verifies that logo against the VMC certificate authorities or using some other reputation system.
  5. If the brand’s BIMI logo was successfully verified, it is attached to the email (for example as a header), before the email is shipped off the recipient’s inbox.
  6. Finally, the recipient looks at her inbox using the mailbox service’s webmail or app, which displays the brand’s logo next to the address and subject in the email listing.
Getting started with BIMI validation on Halon MTA

Halon was the first commercial MTA to add support for many great new security standards such as DMARC, EdDSA DKIM (elliptic curve), DANE and MTA-STS. To our knowledge, we’re the first to add BIMI validation as well. Our BIMI validation module is only 182 lines of Halon script, which I think is pretty impressive (especially considering that it includes a basic ASN.1 parser). To get started with BIMI on Halon MTA, just check out the BIMI module to your configuration repository, add the following to your end-of-DATA script hook. The module uses some X.509 functions from the unreleased Halon MTA 5.8, please let us know if you want to try it out. As for showing the images in your webmail or app, we’re happy to provide guidance and examples.

import $calist from "bimi/ca.crt";
import { bimi, bimi_vmc } from "bimi/bimi.hsl";
import { dmarc } from "dmarc/dmarc.hsl";

$dmarc = dmarc($mail, $senderip, $senderhelo, $senderdomain);
$bimi = bimi($mail, $dmarc);

// Verified Mark Certificate (VMC)
if ($bimi["record"]["a"]) {
	$bimi_vmc = bimi_vmc($bimi, $calist);
	if ($bimi_vmc["indicator"]) {
		$mail->addHeader("BIMI-Indicator",
			str_strip(array_join(pcre_match_all(#/(^(.{0,49})|(.{0,64}))/, 
					$bimi_vmc["indicator"])[1], "\r\n ")),
			["encode" => false]);
	} else
		echo "BIMI VMC error: $bimi_vmc; $bimi";
} else {
	if ($bimi["error"] and $bimi["class"] != "dmarc" and $bimi["class"] != "dns")
		echo "BIMI error: $bimi";
}

Thank you for reading, and don’t hesitate to reach out if you want to learn more!

Halon 5.7 brings DevOps to email engineering with Visual Studio Code

The Halon MTA is the most developer-friendly platform for email transport and security. With version 5.7 “catchy” we’re very happy to introduce an even better experience when using Halon together with Visual Studio Code and Docker Desktop.

Visual Studio Code is one of the most popular source code editor applications, made available for free and open source by Microsoft. It includes many features such as version control with Git, and has tons of available extensions. We have been using one of Visual Studio Code’s fundamental underpinnings, the Monaco code editor, in Halon’s web administration code editor since 2016. In 2018 we released a Visual Studio Code extension for working with both Halon script and YAML configuration files.

More and more customers embrace DevOps processes which combine version control (like Git), provisioning tools (like Chef, Puppet and Ansible) and containers (like Docker). This can improve both the speed and reliability of service development, especially with larger teams and deployments. In order to deliver the best possible experience regardless of which of those technologies our customers’ use, we’re now providing the different Halon components in smaller packages so that they can be integrated into your workflow. For example, you can now have the configuration packer and syntax highlighting in Visual Studio Code, connected to a local Docker instance running the Halon MTA and debugger. The container template enables you to set up a local development environment using Docker Desktop within minutes, complete with all MTA components.

Our extensions deliver syntax highlighting, code completion, built-in documentation, linting (static code analysis), debugging, staging and deployment. One of the most requested features is the debugger, leveraging Visual Studio Code’s breakpoint interface. It can be used on individual modules running inside the Halon script interpreter (hsh), or on a complete deployment by attaching to the MTA process (smtpd) of either a container or VM over SSH. Attaching to the MTA enables you to peek into the full connection and transaction context during not only development and staging, but also during for example troubleshooting of live traffic on production instances. Play the video below to see it in action!

What’s even more powerful is the combination of live debugging with what we call live staging; integrated blue-green testing which allows you deploy a new configuration (complete with all script and modules) for a small, select portion of traffic. It really highlights the unique capabilities which become possible because we develop the MTA, email scripting language and development environment hand-in-hand; as one product with a clear vision.

See the release notes and detailed changelog for a full list of changes. We hope that you will enjoy this new quarterly release. Don’t hesitate to contact us if you want to learn more about what we do, and how we enable email providers to build better services!

More protection with Spamhaus HBL add-on

The Halon MTA provides the widest support for threat protection vendors. It effortlessly integrates with any threat intelligence feed or content filter engine thanks to its scripting language, and we have published several ready-made integrations in our module library. We also sell a bundled license that includes spam, phishing and malware protection from Cyren and Sophos in addition to the MTA platform subscription.

Our integrated package (available as VMs or cloud images) comes with several engines installed, and we just added Spamhaus DQS. It includes support for Spamhaus’ latest developments; the hash blocklist (HBL). It’s a form of anti-virus so that you can filter additional malicious content. Using the HBL in conjunction with IP and domain blocklists, you can secure catch rates of 99.71%* and false positives as low as 0%. You can trial it free for 30 days.

In short, the HBL is a list of cryptographic hashes used to block different elements of malicious email content e.g. a cryptowallet address. Each hash forms a unique string, creating a one-off identifier for that specific piece of content, without using any personally identifiable data. Spamhaus’ HBL includes listings of compromised email addresses, malware files, and crypto wallet addresses. The malware part of the HBL acts like a type of anti-virus protection for your email stream. Typically, to identify malicious email, Spamhaus would assess the sender’s IP and/ or domain reputation. When an email account is from an ESP, like Gmail or Outlook, listings aren’t always possible, otherwise multiple users would be blocked. However, using a hash means Spamhaus can associate malicious email emitting from a specific, compromised email address.

With pioneering collection and distribution methods, Spamhaus’ researchers and engineers hash malware files, listing them in under 20 seconds after detection. By querying the Spamhaus Malware HBL, the return codes will highlight:

  1. If the file is malicious. Here, the file has been assessed by Spamhaus Malware Labs and confirmed as known malware. The return code will also provide the family of malware. 
  2. Or, if the file is suspicious: it has been observed in spam, thereby making it suspicious, though its intent is not confirmed as malicious by Malware labs. Extreme caution should still be taken when handling this file.

Another piece of content used to catch nefarious activity, when IPs and domains can’t be associated with spam, are cryptowallets. Spammers often request payment in cryptocurrencies like Bitcoin and Dash, among others, when trying to extort money from victims. Spamhaus hashes these cryptowallet addresses.  This enables email administrators to easily block the well-known ‘sextortion’ emails. You know the ones…

“I’ve hacked your account. I’ve been watching you visit adult sites. I have made a recording of you from your webcam. Send me money via my Bitcoin address 1N6dubqFmnyQ2DWvi32ppVbc3kKMTYcGW or I’ll share the recording with your contacts”.

To trial the Spamhaus blocklists, including the HBL, sign up for the Spamhaus Data Query Service (DQS). After signing up for the 30-day free trial, you will receive a DQS key. Entering this into the Halon MTA interface will get you protection with IP, Domain, and Hash blocklists. If you are already subscribing to Spamhaus’ IP and domain data, the HBL is available to you free of charge; just tick the HBL box on your Halon interface, and the rest will be done for you!

*Accurate as of March 2021. See here for the latest independent statics by Virus Bulletin.

Halon 5.6 with native plugins and Linux admin

Spring is approaching, and we’re getting ready to ship the next quarterly Halon MTA release, 5.6 “pingy”. For the first time, we’re shipping the web administration interface as separate Linux packages, making it more convenient to setup Halon clusters on Ubuntu and CentOS. This is great news for example if you’re using containers to build an elastic cluster for cost or scalability reasons.

The biggest news however is the new native plugin system, which complements Halon script and FFI. It allows you to add functions to the MTA written in any language compatible with the C library ABI (such as C++ and Go). In addition to adding your own HSL functions, it also enables you to work with the queue and delivery subsystems. Queue plugins can implement things like specific delivery times and customised IP warmup. Delivery plugins can be used to implement for example HTTP delivery to an object storage. Modules should employ the suspend/schedule pattern to avoid blocking, since the Halon MTA is event-based (uses asynchronous IO). This will be described in great detail in an upcoming blog post.

To give you an idea of how it works, consider the example below. It implements a simple sleep function in Go. The Halon_hsl_register function registers a sleep function in HSL. The sleep function itself dispatches a sleepTask, and immediately returns (suspends the script execution). Finally, sleepTask first sleeps, and then wakes the script up again (schedule).

// go build -buildmode c-shared -o sleep.so sleep.go
package main

// #cgo CFLAGS: -I/opt/halon/include
// #cgo LDFLAGS: -Wl,--unresolved-symbols=ignore-all
// #include <HalonMTA.h>
import "C"
import "unsafe"
import "time"

func main() {}
func Halon_version() C.int {
    return C.HALONMTA_PLUGIN_VERSION;
}
func sleepTask(hhc *C.HalonHSLContext, ret *C.HalonHSLValue, sec float64) {
    time.Sleep(time.Duration(sec * 1000.0) * time.Millisecond);
    C.HalonMTA_hsl_schedule(hhc);
}
func sleep(hhc *C.HalonHSLContext, args *C.HalonHSLArguments, ret *C.HalonHSLValue) {
    var t C.double = 0;
    var arg0 = C.HalonMTA_hsl_argument_list(args, 0);
    if (arg0 != nil)
        C.HalonMTA_hsl_value_get(arg0, C.HALONMTA_HSL_TYPE_NUMBER, unsafe.Pointer(&t), nil);
    go sleepTask(hhc, ret, float64(t));
    C.HalonMTA_hsl_suspend_return(ptr); // go can’t use HalonMTA_hsl_suspend()
}
func Halon_hsl_register(hhrc *C.HalonHSLRegisterContext) C.bool {
    C.HalonMTA_hsl_register_function(hhrc, C.CString("sleep"), nil); // go can’t easily get the C function point of an exported method. “Nil” will make the HalonMTA_hsl_register_function use dlsym()
    return true;
}

As usual, the release is packed with other various improvements such as;

  • The FFI now provides the email body (MIME) and other File classes as a C-compatible FILE natively, making it easier to access to the modified email body when integrating with C libraries.
  • The integrated (virtual machine) package now includes Spamhaus DQS which provides additional functionality such as HBL.
  • Support for CSV schemas when using import so that imported CSV files get correctly typed out of the box.
  • Added In-Reply-To header to DSNs to support threaded bounces, which makes it easier and quicker to know which message is associated with what bounce.

See the release notes and detailed changelog for a full list of changes. We hope that you will enjoy this new quarterly release! If you have any questions, or are new to Halon, don’t hesitate to reach out!

Halon 5.5 with DSN extension support

Winter is here, and so is our next quarterly Halon MTA release: 5.5 codename “mappy”. It’s slang for happy on a Monday, which is quite fitting! It comes packed with new features, functions and improvements which opens up even greater possibilities for building differentiated, efficient services.

Our large-scale senders will appreciate additional tools for IP warmup, ensuring deliverability by complying with receiver guidelines. You can now choose the order by which source IPs in a pool are chosen, as opposed to load balancing between all IPs. It allows you to use one IP primarily until it exceeds the threshold for daily warmup for a certain destination. Another feature which is especially useful for senders looking at migrating their email infrastructure to cloud environments (such as AWS) is outbound PROXY support. It enables you to use the powerful HAProxy load balancer rather than NAT, when sharing IPs between MTA instances.

Do you want to leverage the DSN extension to better track email delivery? If so, you’ll appreciate our fully-scriptable RFC 3461 implementation. It allows you to quickly and easily tailor the DSN behaviour exactly to your liking.

The Halon scripting language has been extended with new data storage classes, generator functions with yield, and several new functions for encoding. The strongly-typed Map() and Set() storage classes are performance-optimised alternatives to the more generic array type. Take our revamped DMARC implementation‘s public suffix list lookup for example. It parses a data file, then storing and caching the results in a Set() for efficient lookup of the slightly complex rules:

$rules = memory_fetch("public_suffix_list.dat", function ($k) {
		$rules = Set("string");
		$file = File("file://public_suffix_list.dat");
		if (!$file)
			return false;
		while (is_string($line = $file->readline()))
		{
			$line = str_strip($line);
			if ($line[0] == "/" or $line == "")
				continue;
			$rules->add($line);
		}
		memory_store("public_suffix_list.dat", $rules);
			return $rules;
	});

The integrated (VM) package has received several requested features. First of all, the built-in web server which hosts the web administration and JSON API can now be used to also host custom PHP scripts. It allows you to create additional API endpoints, which can write files used by the Halon script during email processing. One possible use case for this is to receive requests from legal interception systems. Secondly, it is now possible to add local DNS records to the built-in Unbound DNS resolver from the web administration. It can be useful when split-horizon DNS is unavailable, and you need to override some specific DNS record. Finally, it allows you to sort the built-in history on both finished and received time. As usual, the package is based on the latest version of FreeBSD (12.2, which was released a few weeks ago) and comes with updated third-party components.

We hope that you will enjoy this new quarterly release. Please see the release notes for more information, and don’t hesitate to reach out if you have any questions!

Would you like to “like” an email?

Social media and instant messaging (such as Slack) has made reactions like “thumbs up” 👍 and other emojis popular additions to everyday communication. Wouldn’t it be nice if you could quickly press like to confirm or endorse something proposed in an email, rather than having to write a reply?

Dave Crocker, well known for his work on email standards and a senior advisor for M3AAWG, recently submitted the draft “React: Indicating Summary Reaction to a Message”. It introduces email to the world of reaction emojis using an additional email header. It could be implemented in the email clients or webmail (MUAs), and doesn’t require any changes to the transport layer. It’s however an early draft with several considerations yet to be discussed, so I wouldn’t hold my breath just yet.

So, what could it look like? I don’t know, but it didn’t stop me from creating the mockup below!

Halon appoints Anders Långsved new CEO

Halon, the Swedish provider of email security and management solutions, has appointed Anders Långsved as new CEO. With a long background in leading roles in several IT startups and large companies, such as Argogroup (acquired by Ascom), Tele2, Irisity and Wrebit, Anders is now chartered to take Halon to the next levels, on a internationalised market.

Halon now enters a very exciting phase for further expansion where solid product development over the last couple of years has laid a foundation for international success. Anders now has the task to build a team that will enable Halon to realise this potential. Also, in the current environment, we see professional, secure and predictable email becoming even more important than ever before

Richard Berkling, Chairman

Knowing this team for some time now, as a previous board member, it’s great to now be able to work with them every day. Halon has a superb product offering and we are now ready to grow internationally!

Anders Långsved, CEO

Halon offers its secure Mail Transfer Agent to large email providers and senders, world wide. Based in Gothenburg with local development, sales takes to customers such as KPN, One.com and Telia Cygate.

Halon 5.4 with tailored bounces

Image by Dean Hochman

Sometimes you want to customise the bounce generation. It could be anything from translating bounces into another language or making them more user-friendly, to implementing standards such as SRS. In order to do so, you’ve had to override Halon’s default bounce generation. In our new release Halon 5.4 “bouncy”, we’ve added many options to the built-in bounce generator. This enables you to quickly tailor the way bounces look and work, without having to reimplement the default generator.

Queue([
  "delay" => 3600,
  "dsn_delayed" => true,
  "dsn" => [
    "readable_mimepart" => MIME()
      ->setType("text/html")
      ->setBody("<em>choo choo train is delayed</em>")
      ->toString(),
    "original_headers" => false,
    "subject_prepend" => "Not making it in time: ",
    "headers" => [
      "Foo: Bar"
    ]
  ]
]);

As usual, the release comes with many other improvements. The foreign function interface (FFI), which is used for loading external libraries into the MTA executable, can now export File classes as a C++ std::istream and X.509 resources as OpenSSL pointers. This powerful feature enables you to access the (potentially modified) email body as a virtual file. We’ve also added chunking (BDAT) for both sending and receiving, a zero-fill right shift bitwise operator, a data callback to the http() function, FFI function callbacks, and much more. Please see the release notes for a complete list of changes. We hope that you will enjoy this release. If you are new to Halon, don’t hesitate to contact us if you have any questions.

Why concurrency is important for large senders

Photo by Giuseppe Milo

Senders of large email volumes rely on the concept of multiple virtual queues for efficient delivery; separating email from different customers, to different destinations. This guarantees that for example one customer with a gigantic queue, or a destination which is tarpitting, doesn’t affect delivery of other traffic. Email transactions can take some time to complete, often as a result of content filtering at the destination. It is therefore necessary to support many simultaneous (concurrent) connections in each virtual queue, to support the throughput required for timely delivery. If each message takes an average of 1 second to deliver, 50 concurrent connections are needed to deliver 50 email per second.

The total maximum concurrency needed is the number of virtual queues multiplied with the desired concurrency for each queue. Traditional MTAs uses a process- or thread-based connection model. Those models scales poorly, and consequentially struggles to support more than a few thousand concurrent connections per server. This limitation becomes a problem as the system is scaled up, leading to bottlenecks or low customer density per MTA instance.

The Halon MTA uses a modern event-based connection model, sometimes called asynchronous or non-blocking. It scales exceptionally well; supporting tens of thousands of concurrent connections. Multiple worker threads are used to leverage all available CPU cores. This eliminates bottlenecks and can reduce the number of instances needed, which translates to lowered costs and simplified management.

Being asynchronous have benefits for receiving and processing inbound traffic as well. Consider the scenario where thousands of slow, or even idle, clients are connected to an MTA. Traditional MTAs would have to defer new connections from potentially legitimate senders, disturbing the service. Asynchronous MTAs like Halon are on the other hand be able to cope with a significantly large number of concurrent connections, and would handle the scenario above without breaking a sweat.

If you’re interested in learning more, don’t hesitate to reach out.

Halon 5.3 with new powerful queue API

Soap drop by Breic

Since the first Halon MTA release in 2008, we’ve had a text-based queue query syntax called HQL (a play on SQL) as part of our SOAP API. While it has served us well during all those years, it was time to move on to something more modern. The new 5.3 release (codename “buffy”) comes with a Protocol Buffers and JSON API which introduces a programmatic approach to queue operations.

The request and response body schemas are available on our Github page. For your convenience, the QueueList, QueueGroupByQueueUnload and QueueUpdate requests all have the same Condition argument. Those API calls can operate on both the active and defer queues, as well as messages on hold. That is why the condition argument both contains things like retry count, as well as resolved remote MX and IP. You can specify as many conditions as you like, and create logic-or expressions by specifying multiple conditions of the same type. There are exact matching, regular expressions, and intervals for integers and date/time. Needless to say; incredibly powerful.

The QueueGroupBy call returns the distribution based on the grouping parameters and intervals you choose; such as number of messages in various age buckets, grouped by recipient domain. This is useful for getting an overview of a large queue. Queries are blazingly fast, even with very large queues. All queue metadata (essentially the fields available as conditions) is loaded into memory, in order for the virtual sub-queues to work.

Halon 5.3 also comes with a new CLI called halonctl. It happens to be very useful when working with our API, as it can output the API request and response bodies for the command you run in JSON format. As you can se in the example below, the request body is printed first: 

$ halonctl queue update --bounce --state DEFER --jobid foobar --json-request --json
{
    "conditions": {
        "queues": [
            {
                "queue": "DEFER"
                ...

The CLI covers all the functionality of the product, and is a great complement to the web administration. Its configuration management sub-commands are useful for integrating Halon MTA instances into provisioning, deployment and CI/CD toolchains such as Puppet or Chef, where running commands is easier than making API calls.

Halon 5.3 comes with many other great improvements; such as connection pooling, a more efficient queue quota function, a new on-disk queue format and an Iconv() class for internationalisation conversion. Please see the release notes for a complete list of changes. We hope that you will enjoy this release as much as we do! If you are new to Halon, don’t hesitate to contact us, or dig into all our documentation that is available publicly on our website.