Since the first Halon MTA release in 2008, we’ve had a text-based queue query syntax called HQL (a play on SQL) as part of our SOAP API. While it has served us well during all those years, it was time to move on to something more modern. The new 5.3 release (codename “buffy”) comes with a Protocol Buffers and JSON API which introduces a programmatic approach to queue operations.
The request and response body schemas are available on our Github page. For your convenience, the QueueList, QueueGroupBy, QueueUnload and QueueUpdate requests all have the same Condition argument. Those API calls can operate on both the active and defer queues, as well as messages on hold. That is why the condition argument both contains things like retry count, as well as resolved remote MX and IP. You can specify as many conditions as you like, and create logic-or expressions by specifying multiple conditions of the same type. There are exact matching, regular expressions, and intervals for integers and date/time. Needless to say; incredibly powerful.
The QueueGroupBy call returns the distribution based on the grouping parameters and intervals you choose; such as number of messages in various age buckets, grouped by recipient domain. This is useful for getting an overview of a large queue. Queries are blazingly fast, even with very large queues. All queue metadata (essentially the fields available as conditions) is loaded into memory, in order for the virtual sub-queues to work.
Halon 5.3 also comes with a new CLI called halonctl. It happens to be very useful when working with our API, as it can output the API request and response bodies for the command you run in JSON format. As you can se in the example below, the request body is printed first:
The CLI covers all the functionality of the product, and is a great complement to the web administration. Its configuration management sub-commands are useful for integrating Halon MTA instances into provisioning, deployment and CI/CD toolchains such as Puppet or Chef, where running commands is easier than making API calls.
Halon 5.3 comes with many other great improvements; such as connection pooling, a more efficient queue quota function, a new on-disk queue format and an Iconv() class for internationalisation conversion. Please see the release notes for a complete list of changes. We hope that you will enjoy this release as much as we do! If you are new to Halon, don’t hesitate to contact us, or dig into all our documentation that is available publicly on our website.
We’re very proud to announce the upcoming 5.2 release “polly” which introduces a powerful queue policy engine. First and foremost, the queue and SMTP client’s network layer is now asynchronous. This allows an instance to handle tens of thousands of parallel connections. In combination with the reworked connection concurrency limits, this allows dynamic creation of a virtually unlimited number of independent sub-queues. This is useful for senders that need to separate email streams so that those that move slowly or get stuck don’t block others.
As usual, we made it flexible enough to fit any email service provider’s needs. Rather than having a fixed set of parameters and rollup/grouping options for establishing the sub-queues (with their respective thresholds), we allow you to define what constitutes a unique entry. You can choose any combination of fields, and group/rollup entries using regular expressions or wildcard. In the example below, we limit the concurrent per source IP and remote MX, and also rollup all Google’s MX entries into the same entry. The default concurrency is 5, except Google that gets 10.
Sometimes rollup per MX doesn’t cut it. There are several Microsoft Office365 locations (clusters), but the customer MX doesn’t reveal which they are on. To set a certain threshold for Office365 locations, we can rollup and match per MX, but limit per IP, as per the example below. Note that there’s no default threshold; it only affects Office365.
Thresholds and suspensions can be modified on the fly without reloading the configuration via API, CLI, web administration or the MTA itself through this Halon script:
// If we have more than 10 failures per minute, lower rate for 5 minutes
$mx = $arguments["attempt"]["connection"]["remotemx"];
$code = $arguments["attempt"]["result"]["code"];
if ($mx and $code >= 400 and !rate("mx-fail", $mx, 10, 60, ["sync" => false]))
cache ["ttl" => 300]
["remotemx" => $mx],
["rate" => [10, 60]], 300);
The reworked queue naturally comes with many new tools and APIs for interacting with the new functionality. This includes more subtle improvements, like the ability to view the queue’s shape by message age. By pressing an interval, you can dig into the specific messages, which are grouped by fields of your choice.
The new shared memory script functions and API opens up several possibilities. You can script statistic counters, which can then be read periodically over the API. Another use case is pre-loading data into the MTA over the API, rather than fetching and caching from within the script.
The upcoming Halon 5.1 release introduces a new SMTP server proxy script. It’s configured to be executed before specific (or all) SMTP commands, even command which isn’t recognised by the SMTP server. In this blog we’ll describe how to implement our proposed SMTP LANG extension using this new script hook. First of all, we announce the LANG extension and the languages supported in the HELO script:
We celebrate the new year with news on the upcoming release, which bundles many exciting features.
First and foremost, the new RESTful API with an OpenAPI specification makes integration into various development and deployment toolchains much more enjoyable. Since most of our customers already integrate Halon into their directories and control panels by making REST queries from Halon, it makes perfect sense that Halon can be provisioned in the same way.
Secondly, we’re introducing a new end-of-DATA script that’s executed once per message, as opposed to the per-recipient DATA script. Whereas the per-recipient version is convenient when you want to treat each recipient individually and let the Halon software take care of queueing and consolidating the respective actions into an SMTP response, the per-message version gives you maximum flexibility and control over execution. The $transaction variable is populated gradually during the SMTP conversation with sender information, and an array with recipients accepted by each RCPT TO command. To then relay a message to its recipients, you call Queue() for each $transaction["recipients"] and then Accept(). Making per-recipient message modifications using the MIME() class is now easier thanks to the new snapshot() and restore() methods.
The code editor’s built-in CSV editor now supports custom form controls, defined like a “schema” on a per-file basis using a JSON format. You can use checkboxes for booleans, select controllers for enumerated types, and input fields with validation for things like dates, email addresses or any regular expression you like. It makes it much more convenient and safe to create and edit lists and settings that you want to have in your Halon configuration file.
There’s a new LDAP() class that replaces the previous ldap_ functions and LDAP settings in the configuration. It provides greater flexibility, and an improved usage pattern using an iterable LDAP result object.
Finally, there are massive under-the-hood improvements. There’s a new on-disk YAML configurations with JSON schemas and Protobuf control sockets, which is used by the componentised Linux package’s new Visual Studio Code plugin and command line tools. The integrated package is built on FreeBSD 12, which ships with OpenSSL 1.1 and thus TLS 1.3 support. It was published as a standard by IETF in August last year, and is much anticipated as it contains many security improvements over previous TLS versions. The queue database is now using the latest and greatest PostgreSQL version 11.1, and the queue is automatically migrated on boot as usual.
We have that you’ll like this new release as much as we do! Check out the full changelog on GitHub for more information, and familiarise yourself with the important changes outlined in the release notes document before upgrading.
Halon 4.0 introduced a feature we call “live staging” where you can deploy multiple running configurations at the same time, with per-connection conditions. It allows you to reliably rollout changes or new features to a production system for only a few testing IPs, or a select percentage of the traffic. With Halon 4.7, we proudly present “live debugging” using which you can add logpoints to your scripts. It enables you to inspect the full context of SMTP transactions in real-time, using the live staging conditions as connection selector.
Those points are added directly to the Monaco-based IDE, and results are inspected on a per-connection basis. You can create multiple points, triggered by multiple messages, and jump back and forth between them.
You probably know from before that Halon’s scriptable SMTP server enable email providers to avoid blacklisting and increase deliverability. The 4.6 release, “curry”, contains Cyren’s outbound anti-spam (OAS). In combination with our cluster-synchronised rate limit function, it provides incredibly effective and accurate abuse prevention. Just like their Cyren’s inbound anti-spam, OAS uses a hash-sharing technology called recurrent pattern detection (RPD) that identifies outbreak patterns. It’s designed to detect spam from internal sources rather than external, and doesn’t report/contribute any signatures since it could blacklist your own infrastructure.
With the flexibility of scripting you can determine customer/sender identities accurately even in mixed traffic. This is used as identifier for rate limits based on classifiers such as Cyren’s OAS, delivery failure rate, queue size, etc. By using IP source hashing and alternative IPs for suspicious traffic, deferring obvious abuse and controlling connection concurrency, you can achieve high deliverability with minimal administration.
The 4.6 release comes with many additional features and improvements. It adds SNI support to the TLS functions. The Monaco-based code editor now have additional code completion, built-in documentation, tabs, and a mini-map.
In the beginning, everything was ASCII and English. Since then, we’ve seen Unicode (international character sets) and IDN (international domains names) become widely adopted. Last year we implemented SMTPUTF8 that enables international mailboxes.
So why not support other languages in text-based protocols? We give to you “The SMTP Service Extension for Protocol Internationalization” RFC draft, introducing the EHLO keyword LANG. It will be the first SMTP software to support our to-be submitted RFC draft. Initially it will support Swedish, Spanish and Australian, and will default to Swedish when talking to supported systems.
250-LANG SE ES AU
BREV TILL:<hå[email protected]än.se>
250 Togs emot
250 Vi ses!
If you made it this far, April fool! We will publish information on the upcoming 4.6 release some time after the 1st of April.
The main focus in Halon 4.5 release is TLS, hence the name “certy”. Check out the the new features and functions and try them out. Also, the knowledge base is growing with a lot of good how-to’s to help you around.
TLS information has been made accessible in the Halon Platform scripting language, both on the receiving and sending side. Support for X.509 client certificates has been added, allowing you to both verify the sender identity in the SMTP server, as well as identify yourself when sending email through an SMTP client.
Experiment: we configured a busy email system to ask for a client certificate for all inbound connections, and found that approximate 5% of all traffic provides a client identity. Most of the traffic is from Gmail and Office356. We did not collect the percentage of domains, which we leave as an exercise for you.
How to enable this feature and start authenticating clients was documented as KB article.
Implementation and facilitation of TLS reporting (tlsrpt) has begun. It is a new standard for reporting TLS failures, mainly focused on MTA-STS and DANE.
The TLSSocket() class now have a getpeercert() function and the ability to specify a client certificate. Now you see why we called it” certy”?
Support for custom SASL authentication mechanism has been added. This allows you to build authentication schemes such as OTP, OAUTHBEARER or CRAM-MD5, but also EXTERNAL to facilitate the client certificate features. The procedure is documented in our knowledge base along with two sample implementations.
If you haven’t found our knowledge base before, the KB is a place to find how-to’s. The dev team is expanding it as fast as we can, adding topics that customers have asked about.
Finally, I want to highlight the big effort we’ve done to simplify, modernize and overall improve the web administration. This is an ongoing project, and something that we’re paying a lot of attention to. We want to thank, and congratulate, the Bootstrap team for providing such a awesome framework. We managed to get the Bootstrap 4.0 release in, with just a few days of work.
The 4.4 release “lofty” is all about fixing bugs, boost existing features, and improve performance and memory management in the Halon script engine. And like macOS “High Sierra”, it’s fully baked.
The unusually long changelog contains many small improvements. We’ve given the pre/post-delivery script a slight overhaul. It’s now possible to tailor the bounce behaviour via the the SetDNS() function. Additionally, we’ve added $action and $context, as well as functions to set MAIL/RCPT parameters. Finally, the SetSouceIP() enables you to choose an IPv4 and IPv6 address pair, which is a great when you want to provide customers with a private IPv4 and IPv6 or if you want to use diverse address pools.
The improved “Listen on” directive on the Server > SMTP listener page enables more fine-grained control over listen ports and IPs; such as listening on different ports for different IPs.
Quirks and fun trivia
We recently revised our LDAP implementation, and realised that our own syntax and mechanism for failover between hosts is rather superfluous, since OpenLDAP supports that natively. Consequently, we adopted the standard LDAP URI’s in our configuration, and existing configurations will be automatically migrated.
While we support the PROXY protocol (v1) that passes client source IP information from load balancers, we thought it was mostly as HAProxy thing. Apparently, it’s used by many other load balancers such as Amazon ELB, Citrix Netscaler, and F5 BIG-IP. Most of them implements the version 1 (which is human readable), but there is a second version of the protocol that’s binary-packed, and have a quite smart feature: its magic string (protocol identification) is \x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A which translates into literal "\r\n\r\nQUIT\r\n", a string chosen specifically to case an error and disconnect against servers not supporting this protocol. Clever!
If you have a IPv6 only datacenter, but still want to process IPv4 clients, you can do so with a SIIT-DC gateway which uses IPv4-mapped-IPv6 addresses. In Halon, you can use SIIT-DC while still performing IPv4 reputation (such as DNSBL), by extracting and setting the IPv4 address in the CONNECT script. If that doesn’t make the point that we’re very scriptable, then what does?
If you ever had problems signing in to a Halon using Firefox, it can be because a recent change in how “secure cookies” are handled. When signing in over HTTPS, we set the secure cookie flag, which forbids the cookie to be send over a unencrypted HTTP connection to the same host. That is all great, but if you then try to sign in over HTTP (for whatever reason) Firefox will not be able to login because there is already a cookie for that domain with the secure flag and it cannot be replaced, nor accessed. We addressed this by using different cookie names for HTTP and HTTPS. Regardless of this fix, you should not use HTTP when administering your Halon hosts.
Time-of-click protection adds an extra layer of security to protect email users from accessing malicious content. Attacks including malware, ransomware and phishing are becoming more common and more sophisticated with every day, along with users keeping more sensitive information.
With an additional time-of-click protection, Halon will classify links in email every time it’s clicked, before allowing or denying the user to visit it. This means that if the scammer waits two minutes or two months with infecting the site, the user will still be protected when he or she chooses to click the link. It’s the extra layer of security that won’t allow you to visit infected websites by way of a link in an email protected by Halon.
The Halon MTA is a flexible email operations and security platform.
It enables organisations that operate large-scale email services to offer competitive features by rapid implementation
and to lower maintenance costs through reliable deployment and reduced complexity.