Browse

Blog

The past, present and future; TLS for email

Image by Michele M. F.

Let’s start our quick history lesson by setting the scene; Madonna was topping the charts and Kevin Mitnick had just been arrested after nearly three years on the run from the authorities. It’s February 1995 and researchers at Netscape had just released SSL version 2.0, much of the work was credited to Taher Elgamal, often referred to as the father of SSL. SSL version 1.0 was being worked on before that, but was never published.

Things didn’t start out great; SSL 2.0 was quickly discovered to contain a few bad flaws making it vulnerable to man-in-the-middle attacks, amongst other things. It was also impractical to implement especially for web hosting providers since it only supported a fixed domain certificate. SSL 3.0 addressed the issues found in SSL 2.0, and has been in use since 1996. It wasn’t until the discovery of the POODLE attack in 2014 that things started looking grim for SSL 3.0, which was subsequently deprecated in 2015.

TLS 1.0 is basically SSL 3.1, but the protocol was renamed as it was standardised by the IETF. It was first defined in 1999, and the result of a deal struck between Netscape, Microsoft and the IETF. Remember, this was in a time when the browser war between Netscape and Microsoft, and Microsoft had even introduced their own protocol derived from SSL 2.0 only supported in Internet Explorer and their web server IIS. The IETF wanted a standard, and that’s where TLS comes from.

From there on, it’s basically the same story all over again, but with more openness and collaboration. Each TLS version from 1.1 to 1.3 tightened up security, added features and removed old functions. TLS 1.0 and 1.1 were both deprecated in 2020, leaving us with TLS 1.2 and 1.3 as the only versions currently not deprecated.

Impact on email security

Alright, but how does this impact in-transit security for email? Let’s start by talking about SMTP. For server to server communication, opportunistic TLS is what happens in most of the cases. If one (or both) of the servers doesn’t support SSL/TLS the transaction continues unencrypted.

220 mx.halon.io ESMTP // Server introduces itself
EHLO halon.email      // Client sends extended hello
250-mx.halon.io       // Server replies with its capabilities
(cut)
250-STARTTLS          // Informing the client that SSL/TLS is available
(cut)
STARTTLS              // Client asks for the session to be encrypted
220 2.0.0 Ready to start TLS
(handshake and encrypted session)

This by design is vulnerable to man-in-the-middle via downgrade attacks. A malicious party on the line with the ability to modify traffic could very easily force the transaction to continue unencrypted by just dropping the 250-STARTTLS reply. And since there’s no straight forward way of verifying certificates as with HTTPS, it becomes a bit tricky to solve. Two solutions to this problem are DANE and MTA-STS. DANE relies on DNSSEC to establish trust, MTA-STS relies on CA issued certificates.

How to act and what to think about

What TLS protocol versions should you support, since you could argue that even the worst of encryption is better than nothing? Well, Google/Gmail supports TLS 1.2, but also the deprecated 1.0 and 1.1, for inbound and outbound SMTP. Microsoft/Outlook only support TLS 1.2 for both directions. Neither support the older SSL versions. Both will obviously send and receive emails in plain text, if the other party doesn’t support TLS (as is the norm today). To sum things up; make sure that you support TLS 1.2 or higher. Since Microsoft has disabled support for the deprecated TLS 1.0 and 1.1 protocols for server-to-server SMTP, you could probably too. By doing so however, there is a risk that traffic from and to older systems will be sent unencrypted.

More importantly, you should use DANE and/or MTA-STS for your inbound traffic, and validate both DANE and MTA-STS for your outbound traffic. Since these are two competing standards right now, supporting both is probably your best course of action. For sending outbound traffic, Microsoft does MTA-STS checks, whereas Google doesn’t according to our testing. Neither checks DANE, but Microsoft has announced that this will be implemented in December 2021, with inbound DANE coming in December 2022.

When it comes to end-user client communication there’s more protocols to consider in addition to SMTP; mainly IMAP, POP3 and HTTP (for webmail). This is slightly more straightforward, since the user have interactive control over what is happening. Because client connections are generally authenticated with a username and password, it’s a good idea to require TLS if at all possible. This can be done either via implicit TLS; that means running SMTP over port 465, IMAP over port 993 and POP3 (if you still use that) over port 995. The alternative is to use STARTTLS on the default ports, but then it’s important to configure the server software to require TLS to be started, if that’s your policy. Instruct your users to always take extra care to inspect the certificates in use; certificate warnings could be a strong indicator of someone interfering with your traffic.

Black Friday Cyber Monday: Make sure you deliver in time

Image by Mike Cohen

During the busiest shopping season, namely Black Friday Cyber Monday (BFCM), an uptick in email volumes is often a given, but there are a number of things you can do to make sure your email sending performs at its best without getting yourself caught up in the net that is cast to catch the spammers. Are you a brand marketer, or providing email sending services to marketers? Read on!

What does a high volume commercial sender want at BFCM?

As an email sender, you want as much of your email delivered as quickly as possible to the user’s inbox. You want to re-engage as many customers as possible with the best offers. You have been holding back on these loss leaders’ deals for weeks and you want to get the word out. You are not the only one, every e-commerce business and retailer wants the same. Add to this fact that now even B2B SaaS solutions are utilising the BFCM period for promotion and you can see mailbox providers and users have a deluge of email to deal with at this time.

There are plenty of things that you can do over this period to minimise the risk of your email being delayed, rejected, or filtered. I could discuss segmenting your users, and not using BFCM as the time for re-engagement. I could suggest it is not the best time at all for that kind of marketing strategy, but that is not what you will want to hear. You want practical strategies that will protect your reputation, enhance your chances of delivery and protect your deliverability.

What do the postmasters want?

So, we understand what you are looking for as a sender. The postmasters of mailbox providers on the other hand, what are they interested in? They are the folks who have to manage the MTAs and storage servers handling the email you send. Years ago systems could creak under the strain of such an amount of email, and whilst that may still be an issue at some mailbox providers today, modern email software such as Halon MTA handle these spikes in volumes with relative ease. The concern now is primarily for the recipient.

Mailbox providers want to give their users the best possible experience, and spam needs to be managed effectively to achieve that. Despite what you might believe as a marketer, most recipients are pretty laissez-faire about a reasonable proportion of marketing messages getting filtered to spam, even if opted in, if it means their inbox experience is enhanced.

The real dangers with your reputation and deliverability lie not just in the period around BFCM but also in what happens to your reputation the weeks and months after. We know one sure-fire way to do this is to generate complaints. That leads us to what the end-user wants.

What the recipient wants

The receiver of commercial email wants to get relevant, timely information and offers. During the period of BFCM, they have been primed, trained to look out for only extremely high-value offers. There are plenty of times where it is not appropriate to send emails to past customers and assumably you are a sophisticated enough marketer to know how to make those hard decisions about segmenting out some of your user base from offers.

Even if you do everything right during the BFCM period, targeting the right people with the right offer, there are still challenges. Many sending platforms experience issues during these busy days. These issues can lead to delays, and for some senders and recipients, these delays are so severe that the email is delivered even after the offer has expired. As a customer, there is little more infuriating than receiving an incredible offer that is no longer eligible. While this hurts your brand reputation, it certainly hurts your credibility.

How is it even possible that an email you send on Thursday arrives on Sunday? It normally comes down to functionalities in the MTA used by the sender for retrying messages that are not immediately accepted by the recipient’s mailbox provider. Looking at the MTA will help you understand such issues.

Where the issues are for sending services?

Delayed or failed deliveries are almost always due to issues with the email content. Are you a business using an email sending service provider (ESP) to deliver marketing or transactional email? Others using the same service will potentially not be following all the same best practices you are, not keeping up with list hygiene, ensuring stellar permission practices, or even targeting their offers. Once resources are shared, you will potentially be impacted in some way or the other. This is whether you use a shared IP or domain for sending and tracking or not. At the end of the day, the email will all need to go through the same MTAs, and other users’ delays can impact you.

Many ESPs will keep retrying the delivery of your email message for several days if the initial delivery attempt fails. These do not show up as bounces or deliveries during that period in your dashboards. If your email message ends up in such a queue it could well end up being delivered after your offer expires. If your ESP uses a modern and well configured MTA such as Halon, this issue need not exist, as it allows you to configure variable retry times and strategies on a per-campaign or per-message basis. Ask your ESP if they support this feature.

What should a brand marketer do?

So in light of all that we have read and learned in this post, what are the 5 things to remember as an enterprise marketer when it comes to planning BFCM campaigns?

  1. Black Friday/Cyber Monday is not the time to dust off your list and email everyone who has ever purchased something of you in the past 25 years
  2. Run the best possible offer and send only to those who would be interested
  3. Ensure you are not delivering email after the offer expires
  4. Ensure your ESP or sending software can handle expiring delivery of messages in a timely fashion
  5. Do not run flash offers over this period if your ESP is not able to terminate delivery in a timely manner

Now you understand why the issues come up during the BFCM period. You can see that by following the advice above, you can minimise your chances of being filtered or having an email delivered that might tarnish your reputation.

Do you operate MTAs and struggle with roadmap gaps due to inflexibility, or poor delivery performance due to lackluster queue management? Please take a look at how our modern MTA can revolutionise the way you build email infrastructure, and don’t hesitate to reach out.

Discussing email encryption at JPAAWG #4

Photo by Luca Sartoni

Halon is presenting at, and sponsoring, the 4th general meeting of JPAAWG. It’s an independent regional offshoot of the global Messaging, Malware and Mobile Anti-Abuse Working Group backed by long-time M3AAWG members IIJ and TwoFive. M3AAWG is where the industry comes together to work against botnets, malware, spam, viruses, DoS attacks and other online exploitation. It’s the largest industry association, with more than 200 members worldwide, bringing together all the stakeholders in the online community in a confidential, open forum.

I will be speaking about the latest email authentication technologies together with Hackim Farrell from Proofpoint; specifically transport encryption with DANE and MTA-STS. Until recently, there was no widely deployed technology for properly protecting email communication from eavesdropping. We have been advocating for improved email encryption for a long time at conferences and industry meetings all over the globe. We’ve nudged and encouraged, and is very happy to see that there’s now 3 million email domains with DANE, and that both Microsoft and Google has enabled MTA-STS for their consumer mailbox services.

Are you interested in taking part, and happen to know one of the members? If so, I recommend reaching out and applying to attend an upcoming meeting as their guest! If not, we’re happy to guide you through the process. Don’t hesitate to contact us if you want to know more about what we do at M3AAWG, or want to talk about email in general.

The great ARM wrestle; Graviton and RSA

With the introduction of both Apple Silicon (M1) and AWS Graviton, the use of ARM processors has really started to enter the mainstream desktop and server market. Long-term, things are looking really good for ARM in the datacenter, but are we there yet?

The ARM-based Apple Silicon has been praised for delivering good performance with low power usage and heat generation, which is perfect for a portable device. AWS Graviton 2 performs well in a performance to cost perspective as well. Even Microsoft is dipping its toes in the water, so obviously there’s something to this. So is it any good for our Halon MTA? Well, the short answer is that it depends.

Let’s start with the desktop side of things. With our Visual Studio Code extensions it’s very easy to run a local containerised MTA for testing purposes. Some of us here at Halon use Apple laptops with the new M1 chip, and everything works fine in x86 mode under Rosetta 2, but we of course wanted to test it natively! Did it work without issues? Absolutely. Was it exceptional? Yes, in a few ways it was. Benchmarking Halon MTA on Apple M1 against similar x86-based desktops and even Xeon servers reveals excellent per-core performance. DKIM signing and verification performance really stood out, beating the latest (and significantly more power hungry) Intel Xeon processors with the same numbers of cores. However, the workload that you throw at it when testing your configuration and scripts is honestly not enough to really make any change in performance noticeable. In the screenshot below, you can see Halon MTA running natively and containerised on an ARM-based Mac.

Given what you just read about Halon MTA running on Apple’s ARM chips, surely the ARM-based c6g (Graviton 2) instances must be perfect for running Halon MTA on Amazon AWS? Well, here it gets a little complicated. During our first benchmarks of c6g.2xlarge instances (ARM) against c5.2xlarge (traditional x86) we saw a steady performance increase of around 20%, while being roughly 20% cheeper. Fantastic! TLS speeds with AES-128 were also phenomenal, about 30% faster. But that was until we started testing more complex workflows.

When testing isolated DKIM signing with RSA 1024 and EdDSA (elliptic curve), the x86-based c5.2xlarge was about 20% faster than the ARM-based c6g.2xlarge. With RSA 2048, the difference was even bigger; with the x86-based instance being more than twice as fast. Consequentially, when benchmarking a typical email sending setup with DKIM signing (using RSA 2048) and queuing to EBS storage, the signing slowed down the ARM-based c6g.2xlarge instance to the point where it no longer made economical sense. This was verified using OpenSSL’s speed command testing RSA 2048 per-core performance, listed in the table below.

Machine Architecture Sign / s Verify / s
AWS c5.2xlargex861 68556 671
AWS c6g.2xlargeARM29310 720
Apple Mac Mini M1ARM1 76773 035

This might appear very strange, given the exceptional per-core RSA performance of Apple’s M1 chip. Evidently, it’s more about specific CPUs rather than the architecture itself. The following tweet suggests the Amazon AWS team are aware of the RSA performance issues.

It’s possible that this should be attributed to ARM’s Neoverse N1 platform on which Graviton 2 is based, because Cloudflare’s benchmarks of another N1-based CPU called Altra from Ampere seems to have similar RSA performance.

To summarise; ARM can be fast, and is almost certainly here to stay. Is it good for running an MTA server such as Halon? It could be, but it very much depends on the workload, configuration and the type of ARM chip. Fingers crossed for Graviton 3 addressing those RSA issues.

All tests where performed on AWS c5.2xlarge and c6g.2xlarge instances, as well as a Mac Mini with the M1 chip.

Adding BIMI to your mailbox service to improve security and user experience

Image by Valimail

BIMI (Brand Indicators for Message Identification) is an emerging industry standard which attempts to drive adoption of strong email authentication, to help fight phishing, spoofing, and fraudulent emails by displaying verified brand images in the inbox. Several mailbox providers, including Gmail, AOL and Fastmail, are now live with BIMI. Are you looking to increase the security and experience for the users of your mailbox service by implementing this new standard as well? With Halon MTA, you’re ready to get started. Our BIMI module does the heavy lifting; verifying the indicators for inbound email and attaching the images to the email for display in the users’ inbox. Currently there are very few MUAs with BIMI support, but if you develop your own webmail or app the BIMI group has guidance for how to display the verified logo in your user interface.

What BIMI is, and how it works

In short, BIMI displays verified brand images in users’ inboxes, visible even before messages are opened. It builds upon the existing email authentication standards (SPF, DKIM and DMARC), and incentives email senders to embrace those technologies by increasing their brand impression. According to Agari, only 24% of Fortune 500 sending domains are authenticated. BIMI improves visibility and engagement for the brands, while at the same time increasing security for users. It roughly consists of the following steps;

  1. The brand sending the email first needs to implement email authentication (enforcing DMARC) for their domain name.
  2. The brand then creates an appropriate image of their trademarked logo, optionally applies for a Verified Mark Certificate (VMC), stores it on a HTTPS location, and points at it with a BIMI DNS record.
  3. When a BIMI-enabled mailbox service (like Gmail) receives an email from the brand to a specific recipient, the mailbox service’s MTA first verifies that the email is in fact from the sending brand using email authentication like DMARC.
  4. If the email is successfully authenticated, the mailbox service’s MTA checks if the sending brand’s domain has BIMI, and if so, verifies that logo against the VMC certificate authorities or using some other reputation system.
  5. If the brand’s BIMI logo was successfully verified, it is attached to the email (for example as a header), before the email is shipped off the recipient’s inbox.
  6. Finally, the recipient looks at her inbox using the mailbox service’s webmail or app, which displays the brand’s logo next to the address and subject in the email listing.
Getting started with BIMI validation on Halon MTA

Halon was the first commercial MTA to add support for many great new security standards such as DMARC, EdDSA DKIM (elliptic curve), DANE and MTA-STS. To our knowledge, we’re the first to add BIMI validation as well. Our BIMI validation module is only 182 lines of Halon script, which I think is pretty impressive (especially considering that it includes a basic ASN.1 parser). To get started with BIMI on Halon MTA, just check out the BIMI module to your configuration repository, add the following to your end-of-DATA script hook. The module uses some X.509 functions from the unreleased Halon MTA 5.8, please let us know if you want to try it out. As for showing the images in your webmail or app, we’re happy to provide guidance and examples.

import $calist from "bimi/ca.crt";
import { bimi, bimi_vmc } from "bimi/bimi.hsl";
import { dmarc } from "dmarc/dmarc.hsl";

$dmarc = dmarc($mail, $senderip, $senderhelo, $senderdomain);
$bimi = bimi($mail, $dmarc);

// Verified Mark Certificate (VMC)
if ($bimi["record"]["a"]) {
	$bimi_vmc = bimi_vmc($bimi, $calist);
	if ($bimi_vmc["indicator"]) {
		$mail->addHeader("BIMI-Indicator",
			str_strip(array_join(pcre_match_all(#/(^(.{0,49})|(.{0,64}))/, 
					$bimi_vmc["indicator"])[1], "\r\n ")),
			["encode" => false]);
	} else
		echo "BIMI VMC error: $bimi_vmc; $bimi";
} else {
	if ($bimi["error"] and $bimi["class"] != "dmarc" and $bimi["class"] != "dns")
		echo "BIMI error: $bimi";
}

Thank you for reading, and don’t hesitate to reach out if you want to learn more!

Halon 5.7 brings DevOps to email engineering with Visual Studio Code

The Halon MTA is the most developer-friendly platform for email transport and security. With version 5.7 “catchy” we’re very happy to introduce an even better experience when using Halon together with Visual Studio Code and Docker Desktop.

Visual Studio Code is one of the most popular source code editor applications, made available for free and open source by Microsoft. It includes many features such as version control with Git, and has tons of available extensions. We have been using one of Visual Studio Code’s fundamental underpinnings, the Monaco code editor, in Halon’s web administration code editor since 2016. In 2018 we released a Visual Studio Code extension for working with both Halon script and YAML configuration files.

More and more customers embrace DevOps processes which combine version control (like Git), provisioning tools (like Chef, Puppet and Ansible) and containers (like Docker). This can improve both the speed and reliability of service development, especially with larger teams and deployments. In order to deliver the best possible experience regardless of which of those technologies our customers’ use, we’re now providing the different Halon components in smaller packages so that they can be integrated into your workflow. For example, you can now have the configuration packer and syntax highlighting in Visual Studio Code, connected to a local Docker instance running the Halon MTA and debugger. The container template enables you to set up a local development environment using Docker Desktop within minutes, complete with all MTA components.

Our extensions deliver syntax highlighting, code completion, built-in documentation, linting (static code analysis), debugging, staging and deployment. One of the most requested features is the debugger, leveraging Visual Studio Code’s breakpoint interface. It can be used on individual modules running inside the Halon script interpreter (hsh), or on a complete deployment by attaching to the MTA process (smtpd) of either a container or VM over SSH. Attaching to the MTA enables you to peek into the full connection and transaction context during not only development and staging, but also during for example troubleshooting of live traffic on production instances. Play the video below to see it in action!

What’s even more powerful is the combination of live debugging with what we call live staging; integrated blue-green testing which allows you deploy a new configuration (complete with all script and modules) for a small, select portion of traffic. It really highlights the unique capabilities which become possible because we develop the MTA, email scripting language and development environment hand-in-hand; as one product with a clear vision.

See the release notes and detailed changelog for a full list of changes. We hope that you will enjoy this new quarterly release. Don’t hesitate to contact us if you want to learn more about what we do, and how we enable email providers to build better services!

More protection with Spamhaus HBL add-on

The Halon MTA provides the widest support for threat protection vendors. It effortlessly integrates with any threat intelligence feed or content filter engine thanks to its scripting language, and we have published several ready-made integrations in our module library. We also sell a bundled license that includes spam, phishing and malware protection from Cyren and Sophos in addition to the MTA platform subscription.

Our integrated package (available as VMs or cloud images) comes with several engines installed, and we just added Spamhaus DQS. It includes support for Spamhaus’ latest developments; the hash blocklist (HBL). It’s a form of anti-virus so that you can filter additional malicious content. Using the HBL in conjunction with IP and domain blocklists, you can secure catch rates of 99.71%* and false positives as low as 0%. You can trial it free for 30 days.

In short, the HBL is a list of cryptographic hashes used to block different elements of malicious email content e.g. a cryptowallet address. Each hash forms a unique string, creating a one-off identifier for that specific piece of content, without using any personally identifiable data. Spamhaus’ HBL includes listings of compromised email addresses, malware files, and crypto wallet addresses. The malware part of the HBL acts like a type of anti-virus protection for your email stream. Typically, to identify malicious email, Spamhaus would assess the sender’s IP and/ or domain reputation. When an email account is from an ESP, like Gmail or Outlook, listings aren’t always possible, otherwise multiple users would be blocked. However, using a hash means Spamhaus can associate malicious email emitting from a specific, compromised email address.

With pioneering collection and distribution methods, Spamhaus’ researchers and engineers hash malware files, listing them in under 20 seconds after detection. By querying the Spamhaus Malware HBL, the return codes will highlight:

  1. If the file is malicious. Here, the file has been assessed by Spamhaus Malware Labs and confirmed as known malware. The return code will also provide the family of malware. 
  2. Or, if the file is suspicious: it has been observed in spam, thereby making it suspicious, though its intent is not confirmed as malicious by Malware labs. Extreme caution should still be taken when handling this file.

Another piece of content used to catch nefarious activity, when IPs and domains can’t be associated with spam, are cryptowallets. Spammers often request payment in cryptocurrencies like Bitcoin and Dash, among others, when trying to extort money from victims. Spamhaus hashes these cryptowallet addresses.  This enables email administrators to easily block the well-known ‘sextortion’ emails. You know the ones…

“I’ve hacked your account. I’ve been watching you visit adult sites. I have made a recording of you from your webcam. Send me money via my Bitcoin address 1N6dubqFmnyQ2DWvi32ppVbc3kKMTYcGW or I’ll share the recording with your contacts”.

To trial the Spamhaus blocklists, including the HBL, sign up for the Spamhaus Data Query Service (DQS). After signing up for the 30-day free trial, you will receive a DQS key. Entering this into the Halon MTA interface will get you protection with IP, Domain, and Hash blocklists. If you are already subscribing to Spamhaus’ IP and domain data, the HBL is available to you free of charge; just tick the HBL box on your Halon interface, and the rest will be done for you!

*Accurate as of March 2021. See here for the latest independent statics by Virus Bulletin.

Halon 5.6 with native plugins and Linux admin

Spring is approaching, and we’re getting ready to ship the next quarterly Halon MTA release, 5.6 “pingy”. For the first time, we’re shipping the web administration interface as separate Linux packages, making it more convenient to setup Halon clusters on Ubuntu and CentOS. This is great news for example if you’re using containers to build an elastic cluster for cost or scalability reasons.

The biggest news however is the new native plugin system, which complements Halon script and FFI. It allows you to add functions to the MTA written in any language compatible with the C library ABI (such as C++ and Go). In addition to adding your own HSL functions, it also enables you to work with the queue and delivery subsystems. Queue plugins can implement things like specific delivery times and customised IP warmup. Delivery plugins can be used to implement for example HTTP delivery to an object storage. Modules should employ the suspend/schedule pattern to avoid blocking, since the Halon MTA is event-based (uses asynchronous IO). This will be described in great detail in an upcoming blog post.

To give you an idea of how it works, consider the example below. It implements a simple sleep function in Go. The Halon_hsl_register function registers a sleep function in HSL. The sleep function itself dispatches a sleepTask, and immediately returns (suspends the script execution). Finally, sleepTask first sleeps, and then wakes the script up again (schedule).

// go build -buildmode c-shared -o sleep.so sleep.go
package main

// #cgo CFLAGS: -I/opt/halon/include
// #cgo LDFLAGS: -Wl,--unresolved-symbols=ignore-all
// #include <HalonMTA.h>
import "C"
import "unsafe"
import "time"

func main() {}
func Halon_version() C.int {
    return C.HALONMTA_PLUGIN_VERSION;
}
func sleepTask(hhc *C.HalonHSLContext, ret *C.HalonHSLValue, sec float64) {
    time.Sleep(time.Duration(sec * 1000.0) * time.Millisecond);
    C.HalonMTA_hsl_schedule(hhc);
}
func sleep(hhc *C.HalonHSLContext, args *C.HalonHSLArguments, ret *C.HalonHSLValue) {
    var t C.double = 0;
    var arg0 = C.HalonMTA_hsl_argument_list(args, 0);
    if (arg0 != nil)
        C.HalonMTA_hsl_value_get(arg0, C.HALONMTA_HSL_TYPE_NUMBER, unsafe.Pointer(&t), nil);
    go sleepTask(hhc, ret, float64(t));
    C.HalonMTA_hsl_suspend_return(ptr); // go can’t use HalonMTA_hsl_suspend()
}
func Halon_hsl_register(hhrc *C.HalonHSLRegisterContext) C.bool {
    C.HalonMTA_hsl_register_function(hhrc, C.CString("sleep"), nil); // go can’t easily get the C function point of an exported method. “Nil” will make the HalonMTA_hsl_register_function use dlsym()
    return true;
}

As usual, the release is packed with other various improvements such as;

  • The FFI now provides the email body (MIME) and other File classes as a C-compatible FILE natively, making it easier to access to the modified email body when integrating with C libraries.
  • The integrated (virtual machine) package now includes Spamhaus DQS which provides additional functionality such as HBL.
  • Support for CSV schemas when using import so that imported CSV files get correctly typed out of the box.
  • Added In-Reply-To header to DSNs to support threaded bounces, which makes it easier and quicker to know which message is associated with what bounce.

See the release notes and detailed changelog for a full list of changes. We hope that you will enjoy this new quarterly release! If you have any questions, or are new to Halon, don’t hesitate to reach out!

Not all successful DSN deploys are successful

Our latest release 5.5 adds support for the RFC 3461 DSN extension, and published a guide on how to get started. At first glance it may seem like a very nifty feature, and one may wonder why it’s not just enabled by default. Fact it, the DNS extension comes with a few peculiarities which need to be considered.

In many email environments, the receiver has multiple store-and-forward MTAs before the message reaches the recipient’s inbox. Each MTA in the path has a different purpose, and may reject the message (if it is spam, for example) rather than deliver it forward. This is what requires MTAs to send “failed” DSN notifications; when an MTA which isn’t your first (edge) rejects a message. The sender always has to be notified as per the transaction safety requirement of email.

One of the features provided by the DSN extension is that you may also request “delivered” notifications by the last MTA before the recipient’s inbox (or “relayed” notifications by the last MTA supporting the DSN extension). This lets the sender know when and if the message was delivered.

Thinking about it, this feature may be nice to have enabled in your MTA for outbound email. It would let you or your users know if and when a message is delivered; rather than only notifying you when and if delivery fails.

If you implement this feature for inbound traffic however, it would be one of your own MTA that would have to send those notifications out to anyone on the Internet that requested them. The notifications would go out from the last MTA in your inbound path. When messages are processed and routed internally, additional information that should not be visible to the sender are sometimes added. Allowing the sender to request a notification possibly containing some of this possibly sensitive information might not be ideal. This concern has be raised many times before, for example in the Exim documentation

Note: Supplying success-DSN messages has been criticised on privacy grounds; it can leak details of internal forwarding.

During our implementation and testing we enabled success notifications for some of our outbound traffic, to see which mailbox providers send such notifications. We were quite surprised to learn that Microsoft includes quite a lot of information regarding the recipients in their success notifications. We instantly reported this to the MSRC team but it was rejected as not being an issue 🙈

Amongst those hundred of lines of X-MS- headers you receive in their success notifications, I’m not sure if the last login time of the recipient is necessary. This is in fact the actual time when the recipient (eg. [email protected]) last checked their email:

X-MS-UserLastLogonTime: 10/1/2020 12:06:31 AM

Bottom line is; should you choose to implement this feature, make sure you do not add sensitive or unnecessary information to your success notifications that may expose your network internals or your customers’ personal information.

Halon 5.5 with DSN extension support

Winter is here, and so is our next quarterly Halon MTA release: 5.5 codename “mappy”. It’s slang for happy on a Monday, which is quite fitting! It comes packed with new features, functions and improvements which opens up even greater possibilities for building differentiated, efficient services.

Our large-scale senders will appreciate additional tools for IP warmup, ensuring deliverability by complying with receiver guidelines. You can now choose the order by which source IPs in a pool are chosen, as opposed to load balancing between all IPs. It allows you to use one IP primarily until it exceeds the threshold for daily warmup for a certain destination. Another feature which is especially useful for senders looking at migrating their email infrastructure to cloud environments (such as AWS) is outbound PROXY support. It enables you to use the powerful HAProxy load balancer rather than NAT, when sharing IPs between MTA instances.

Do you want to leverage the DSN extension to better track email delivery? If so, you’ll appreciate our fully-scriptable RFC 3461 implementation. It allows you to quickly and easily tailor the DSN behaviour exactly to your liking.

The Halon scripting language has been extended with new data storage classes, generator functions with yield, and several new functions for encoding. The strongly-typed Map() and Set() storage classes are performance-optimised alternatives to the more generic array type. Take our revamped DMARC implementation‘s public suffix list lookup for example. It parses a data file, then storing and caching the results in a Set() for efficient lookup of the slightly complex rules:

$rules = memory_fetch("public_suffix_list.dat", function ($k) {
		$rules = Set("string");
		$file = File("file://public_suffix_list.dat");
		if (!$file)
			return false;
		while (is_string($line = $file->readline()))
		{
			$line = str_strip($line);
			if ($line[0] == "/" or $line == "")
				continue;
			$rules->add($line);
		}
		memory_store("public_suffix_list.dat", $rules);
			return $rules;
	});

The integrated (VM) package has received several requested features. First of all, the built-in web server which hosts the web administration and JSON API can now be used to also host custom PHP scripts. It allows you to create additional API endpoints, which can write files used by the Halon script during email processing. One possible use case for this is to receive requests from legal interception systems. Secondly, it is now possible to add local DNS records to the built-in Unbound DNS resolver from the web administration. It can be useful when split-horizon DNS is unavailable, and you need to override some specific DNS record. Finally, it allows you to sort the built-in history on both finished and received time. As usual, the package is based on the latest version of FreeBSD (12.2, which was released a few weeks ago) and comes with updated third-party components.

We hope that you will enjoy this new quarterly release. Please see the release notes for more information, and don’t hesitate to reach out if you have any questions!