For those of you working with larger amounts of traffic, which solution is better; building an email platform from scratch, or using an email security and delivery platform (ESDP) service? In this blog post, we describe both solutions so you can determine which one suits your company best.
We’re very proud to announce the upcoming 5.2 release “polly” which introduces a powerful queue policy engine. First and foremost, the queue and SMTP client’s network layer is now asynchronous. This allows an instance to handle tens of thousands of parallel connections. In combination with the reworked connection concurrency limits, this allows dynamic creation of a virtually unlimited number of independent sub-queues. This is useful for senders that need to separate email streams so that those that move slowly or get stuck don’t block others.
As usual, we made it flexible enough to fit any email service provider’s needs. Rather than having a fixed set of parameters and rollup/grouping options for establishing the sub-queues (with their respective thresholds), we allow you to define what constitutes a unique entry. You can choose any combination of fields, and group/rollup entries using regular expressions or wildcard. In the example below, we limit the concurrent per source IP and remote MX, and also rollup all Google’s MX entries into the same entry. The default concurrency is 5, except Google that gets 10.
Sometimes rollup per MX doesn’t cut it. There are several Microsoft Office365 locations (clusters), but the customer MX doesn’t reveal which they are on. To set a certain threshold for Office365 locations, we can rollup and match per MX, but limit per IP, as per the example below. Note that there’s no default threshold; it only affects Office365.
Thresholds and suspensions can be modified on the fly without reloading the configuration via API, CLI, web administration or the MTA itself through this Halon script:
// If we have more than 10 failures per minute, lower rate for 5 minutes
$mx = $arguments["attempt"]["connection"]["remotemx"];
$code = $arguments["attempt"]["result"]["code"];
if ($mx and $code >= 400 and !rate("mx-fail", $mx, 10, 60, ["sync" => false]))
cache ["ttl" => 300]
["remotemx" => $mx],
["rate" => [10, 60]], 300);
The reworked queue naturally comes with many new tools and APIs for interacting with the new functionality. This includes more subtle improvements, like the ability to view the queue’s shape by message age. By pressing an interval, you can dig into the specific messages, which are grouped by fields of your choice.
The new shared memory script functions and API opens up several possibilities. You can script statistic counters, which can then be read periodically over the API. Another use case is pre-loading data into the MTA over the API, rather than fetching and caching from within the script.
If you’re operating an email delivery platform that’s growing in traffic but isn’t operating at its full potential, you might want to look for other options. There are of course challenges associated with switching platforms, but in the end, it can prove to be the best solution for your company.
The scripting on a platform created specifically for email enables you to create more with much less code. If you’re accustomed to a home-brew system based on open source components and decide to evaluate a comprehensive and scriptable email platform, you’re gonna find yourself spending less time on development and more time on value-adding strategies.
The upcoming Halon 5.1 release introduces a new SMTP server proxy script. It’s configured to be executed before specific (or all) SMTP commands, even command which isn’t recognised by the SMTP server. In this blog we’ll describe how to implement our proposed SMTP LANG extension using this new script hook. First of all, we announce the LANG extension and the languages supported in the HELO script:
Deployment of changes or new features can be challenging, time-consuming and risky. Many service providers don’t have the infrastructure for production testing.
One great way of quickly and safely rolling out changes is blue-green deployment. Halon provides built-in, integrated traffic splitting that we call live staging. It’s a unique method that allows you to try out new code and configuration on a production host for only a selected part of your traffic, selected by random or IP address. This creates two virtual environments. The code editor and tooling makes it easy to test the working copy before you decide to deploy.
If you are using an on-prem, open source based solution for your MTA, you have the advantage of low initial investment costs with no license cost. But later on, your operating costs can start to become quite severe. Factors that can become quite expensive for an email service provider is server rents or costs for hardware, as well as various costs for updates, maintenance and development. Additional support costs can reside depending on the setup. You may also experience issues with complicated pathways, and various components may be coupled together in complex ways.
As a supporting member, we’re happy to be participating in our 7th meeting on February 18-21. M3AAWG meetings are an exceptional opportunity to discuss the latest in messaging security with other professionals in a focused environment of working sessions and educational panels.
We would be delighted if you joined the transport encryption session that I’m speaking at, on Wednesday. Also, if you want to meet up, just get in touch! We’re all around; product, sales and engineering.
We celebrate the new year with news on the upcoming release, which bundles many exciting features.
First and foremost, the new RESTful API with an OpenAPI specification makes integration into various development and deployment toolchains much more enjoyable. Since most of our customers already integrate Halon into their directories and control panels by making REST queries from Halon, it makes perfect sense that Halon can be provisioned in the same way.
Secondly, we’re introducing a new end-of-DATA script that’s executed once per message, as opposed to the per-recipient DATA script. Whereas the per-recipient version is convenient when you want to treat each recipient individually and let the Halon software take care of queueing and consolidating the respective actions into an SMTP response, the per-message version gives you maximum flexibility and control over execution. The $transaction variable is populated gradually during the SMTP conversation with sender information, and an array with recipients accepted by each RCPT TO command. To then relay a message to its recipients, you call Queue() for each $transaction["recipients"] and then Accept(). Making per-recipient message modifications using the MIME() class is now easier thanks to the new snapshot() and restore() methods.
The code editor’s built-in CSV editor now supports custom form controls, defined like a “schema” on a per-file basis using a JSON format. You can use checkboxes for booleans, select controllers for enumerated types, and input fields with validation for things like dates, email addresses or any regular expression you like. It makes it much more convenient and safe to create and edit lists and settings that you want to have in your Halon configuration file.
There’s a new LDAP() class that replaces the previous ldap_ functions and LDAP settings in the configuration. It provides greater flexibility, and an improved usage pattern using an iterable LDAP result object.
Finally, there are massive under-the-hood improvements. There’s a new on-disk YAML configurations with JSON schemas and Protobuf control sockets, which is used by the componentised Linux package’s new Visual Studio Code plugin and command line tools. The integrated package is built on FreeBSD 12, which ships with OpenSSL 1.1 and thus TLS 1.3 support. It was published as a standard by IETF in August last year, and is much anticipated as it contains many security improvements over previous TLS versions. The queue database is now using the latest and greatest PostgreSQL version 11.1, and the queue is automatically migrated on boot as usual.
We have that you’ll like this new release as much as we do! Check out the full changelog on GitHub for more information, and familiarise yourself with the important changes outlined in the release notes document before upgrading.
The Halon MTA is a flexible email operations and security platform.
It enables organisations that operate large-scale email services to offer competitive features by rapid implementation
and to lower maintenance costs through reliable deployment and reduced complexity.