For those of you working with larger amounts of traffic, which solution is better; building an email platform from scratch, or using an email security and delivery platform (ESDP) service? In this blog post, we describe both solutions so you can determine which one suits your company best.
For many companies, an MTA is simply something required for handling your email processes. Some might look for the cheapest solution, others that seem to be the easiest to implement as soon as possible. Regardless of your needs, you want to look for a solution suitable for the volume of emails you handle, and the scalability you might experience. Here are some pros and cons of solutions suitable for larger amounts of traffic.
An on-premise MTA is a software that sends and receives emails, usually open-source software, running on the company’s servers. When running an on-premise MTA, a few or just one person is responsible for the platform and has all knowledge about the platform and all its functions.
The license cost, usually based on open source, is low.
You get the flexibility to customize the platform as you like.
You get full control over the software.
Running an on-premise MTA may lead to problems with scalability and performance.
There are comprehensive hardware requirements.
The costs for development, servers, and maintenance are high.
Time-consuming procedures like support are required.
Email security and delivery platform (ESPD)
An ESDP MTA is an email platform that functions as a smart combination of both worlds; it has the flexibility of building a platform yourself with open source software, and the advantages that a commercially supported product brings.
You get the flexibility to customize the platform to your needs and requirements.
Both server cost and time-to-market are reduced.
Less development and maintenance are required compared to other solutions, and updates are continuously enhancing your platform.
The ease to scale up as traffic increases.
This solution is not suitable for small amounts of traffic.
Thinking about implementing a new email platform, but keep postponing the process since it seems way too time-consuming? Here we describe the whole process to show you know how fast it can be done!
After you have decided that your company needs a new email platform, the routine usually looks the same. It goes from requirements to implementation and through testing, evaluation, and verification of the operational production system. Nevertheless, the duration of this process varies from weeks to months.
Email service providers have different needs, so obviously, one size does not fit all. Some want a service that filters their emails from spam and viruses. Other companies want to consolidate all their incoming and outgoing emails for their hosting environment (mailbox). Others, often internet service providers (ISPs), want it for their infrastructure. Let us go through the entire process.
The customer provides a requirement specification to the supplier, who goes through it thoroughly to make sure everything specified can be fulfilled and delivered. In general, everything can be done, but in some cases, to comply, additional 3rd party solutions might be needed.
This stage is usually pretty swift, but it depends on the customer (for instance, public procurement usually takes a bit longer). Difficulties that may arise during this part of the process could be answering the customer’s questions and meeting the new platform demands, as well as documenting the system to be replaced and its requirements.
During the implementation, product specialists work with installing and configuring the new email platform based on the agreed requirements, in close collaboration with the customer. Meetings are continuously held with the customer to make sure everything is working smoothly. Also, information from the old email platform is analyzed and secured to be used for integration during the implementation.
Also, during the implementation, the platform is adjusted to suit the customer’s requirements, and if problems occur, the supplier makes suitable improvements to solve the.
3 Testing, evaluating and verifying
After implementation, it is up to the customer to test, evaluate, and verify the new platform and its functions and asses to what level the agreed requirements are met. In this part of the process, minor adjustments of the configuration might be needed, and IP addresses may need to be warmed.
Also, live traffic then needs to be tested. It begins with the customer directing some of the traffic from the old platform to the new one (if possible). Should a problem occur, or changes are to be made during live operation, many choose to use blue-green deployment.
At Halon, our live staging function is an enhanced way of using blue-green deployment. With the regular procedure, you create two identical environments to test the traffic. With live staging, you can run two similar configurations in the same environment, and try only a small portion of the traffic using the new configuration, to make sure that the testing is safe. If everything is operating as it should, you can then gradually increase the amount of traffic, until all traffic has been moved to the new configuration.
4 Deploying the new platform
If everything goes according to plan, and the customer appreciates the new system, you celebrate and go-to-market!
We’re very proud to announce the upcoming 5.2 release “polly” which introduces a powerful queue policy engine. First and foremost, the queue and SMTP client’s network layer is now asynchronous. This allows an instance to handle tens of thousands of parallel connections. In combination with the reworked connection concurrency limits, this allows dynamic creation of a virtually unlimited number of independent sub-queues. This is useful for senders that need to separate email streams so that those that move slowly or get stuck don’t block others.
As usual, we made it flexible enough to fit any email service provider’s needs. Rather than having a fixed set of parameters and rollup/grouping options for establishing the sub-queues (with their respective thresholds), we allow you to define what constitutes a unique entry. You can choose any combination of fields, and group/rollup entries using regular expressions or wildcard. In the example below, we limit the concurrent per source IP and remote MX, and also rollup all Google’s MX entries into the same entry. The default concurrency is 5, except Google that gets 10.
Sometimes rollup per MX doesn’t cut it. There are several Microsoft Office365 locations (clusters), but the customer MX doesn’t reveal which they are on. To set a certain threshold for Office365 locations, we can rollup and match per MX, but limit per IP, as per the example below. Note that there’s no default threshold; it only affects Office365.
API, CLI, web administration or the MTA itself can, through this Halon script, add suspensions, and threshold can be adjusted dynamically:
// If we have more than 10 failures per minute, lower rate for 5 minutes
$mx = $arguments["attempt"]["connection"]["remotemx"];
$code = $arguments["attempt"]["result"]["code"];
if ($mx and $code >= 400 and !rate("mx-fail", $mx, 10, 60, ["sync" => false]))
cache ["ttl" => 300]
["remotemx" => $mx],
["rate" => [10, 60]], 300);
The reworked queue naturally comes with many new tools and APIs for interacting with the new functionality. This includes more subtle improvements, like the ability to view the queue’s shape by message age. By pressing an interval, you can dig into the specific messages, which are grouped by fields of your choice.
The new shared memory script functions and API opens up several possibilities. You can script statistic counters, which can then be read periodically over the API. Another use case is pre-loading data into the MTA over the API, rather than fetching and caching from within the script.
If you’re operating an email delivery platform that’s growing in traffic but isn’t operating at its full potential, you might want to look for other options. There are of course challenges associated with switching platforms, but in the end, it can prove to be the best solution for your company.
The scripting on a platform created specifically for email enables you to create more with much less code. If you’re accustomed to a home-brew system based on open source components and decide to evaluate a comprehensive and scriptable email platform, you’re gonna find yourself spending less time on development and more time on value-adding strategies.
The upcoming Halon 5.1 release introduces a new SMTP server proxy script. It’s configured to be executed before specific (or all) SMTP commands, even command which isn’t recognised by the SMTP server. In this blog we’ll describe how to implement our proposed SMTP LANG extension using this new script hook. First of all, we announce the LANG extension and the languages supported in the HELO script:
Deployment of changes or new features can be challenging, time-consuming and risky. Many service providers don’t have the infrastructure for production testing.
One great way of quickly and safely rolling out changes is blue-green deployment. Halon provides built-in, integrated traffic splitting that we call live staging. It’s a unique method that allows you to try out new code and configuration on a production host for only a selected part of your traffic, selected by random or IP address. This creates two virtual environments. The code editor and tooling makes it easy to test the working copy before you decide to deploy.
If you are using an on-prem, open source based solution for your MTA, you have the advantage of low initial investment costs with no license cost. But later on, your operating costs can start to become quite severe. Factors that can become quite expensive for an email service provider is server rents or costs for hardware, as well as various costs for updates, maintenance and development. Additional support costs can reside depending on the setup. You may also experience issues with complicated pathways, and various components may be coupled together in complex ways.
As a supporting member, we’re happy to be participating in our 7th meeting on February 18-21. M3AAWG meetings are an exceptional opportunity to discuss the latest in messaging security with other professionals in a focused environment of working sessions and educational panels.
We would be delighted if you joined the transport encryption session that I’m speaking at, on Wednesday. Also, if you want to meet up, just get in touch! We’re all around; product, sales and engineering.
We celebrate the new year with news on the upcoming release, which bundles many exciting features.
First and foremost, the new RESTful API with an OpenAPI specification makes integration into various development and deployment toolchains much more enjoyable. Since most of our customers already integrate Halon into their directories and control panels by making REST queries from Halon, it makes perfect sense that Halon can be provisioned in the same way.
Secondly, we’re introducing a new end-of-DATA script that’s executed once per message, as opposed to the per-recipient DATA script. Whereas the per-recipient version is convenient when you want to treat each recipient individually and let the Halon software take care of queueing and consolidating the respective actions into an SMTP response, the per-message version gives you maximum flexibility and control over execution. The $transaction variable is populated gradually during the SMTP conversation with sender information, and an array with recipients accepted by each RCPT TO command. To then relay a message to its recipients, you call Queue() for each $transaction["recipients"] and then Accept(). Making per-recipient message modifications using the MIME() class is now easier thanks to the new snapshot() and restore() methods.
The code editor’s built-in CSV editor now supports custom form controls, defined like a “schema” on a per-file basis using a JSON format. You can use checkboxes for booleans, select controllers for enumerated types, and input fields with validation for things like dates, email addresses or any regular expression you like. It makes it much more convenient and safe to create and edit lists and settings that you want to have in your Halon configuration file.
There’s a new LDAP() class that replaces the previous ldap_ functions and LDAP settings in the configuration. It provides greater flexibility, and an improved usage pattern using an iterable LDAP result object.
Finally, there are massive under-the-hood improvements. There’s a new on-disk YAML configurations with JSON schemas and Protobuf control sockets, which is used by the componentised Linux package’s new Visual Studio Code plugin and command line tools. The integrated package is built on FreeBSD 12, which ships with OpenSSL 1.1 and thus TLS 1.3 support. It was published as a standard by IETF in August last year, and is much anticipated as it contains many security improvements over previous TLS versions. The queue database is now using the latest and greatest PostgreSQL version 11.1, and the queue is automatically migrated on boot as usual.
We have that you’ll like this new release as much as we do! Check out the full changelog on GitHub for more information, and familiarise yourself with the important changes outlined in the release notes document before upgrading.
The Halon MTA is a flexible email operations and security platform.
It enables organisations that operate large-scale email services to offer competitive features by rapid implementation
and to lower maintenance costs through reliable deployment and reduced complexity.