Microsoft revealed a project called “Monaco” in November 2013 as a part of the online version of Visual Studio. A few years later, the open source Visual Studio Code editor was released, which also used it as foundation. Finally, in June 2016, Microsoft released a standalone version of the Monaco Editor on GitHub.
We’re very happy to announce that the Monaco integration is part of our Halon 3.5-r3 release that we shipped in early August (2016). The Ace editor (by Cloud9) have served us very well during many years, and remains an important part of our product as it continues to be used in for example the “flow chart” block’s scripting area.
Finally we like to thank Alexandru Dima and everybody else who has been working on Monaco for this excellent piece of software.
Today’s leading spam filter technologies offer a very high degree of accuracy. In this blog I’ll describe the current state of spam classification, and propose a pretty innovative method that can significantly improve both senders’ and recipients’ satisfaction (as well as reducing the burden on administrators and support staff) by enabling senders to report false positives if they pass a CAPTCHA test. Let’s start by familiarising ourselves with the history of anti-spam.
The terminology that we normally use is
False positive, a missed spam that slipped through filters into a user’s mail box
False negative, a blocked desired (legitimate) email (“ham”)
Historically, spam filters had poor accuracy and low performance, and email was scanned after being accepted (probably as a consequence of the former). Finding themselves unable to reject email, they offered actions such as putting suspected spam in a junk folder, quarantine or by tagging the subject line.
This I believe, significantly damaged people’s trust in email as a reliable transport, simply because it makes legitimate (potentially important) email disappear.
The leading spam classification technologies today however, offers both high accuracy and performance. Many of them, including Cyren (that we use), uses fuzzy checksums (or “patterns”) to measure and classify email in a distributed, collaborative fashion. By constantly updating the hashing logic, anti-spam vendors are able to adopt as spammers evolve their tactics. By primarily looking at individual spam “outbreaks”, the false positive ratio is generally low in such systems. This is key, since people tend to be much less bothered by a few false negatives (missed spam) rather than having desired email blocked.
The high accuracy and performance also makes rejecting spam (rather than accepting it) a viable option. Rejecting spam is arguable superior to accepting and quarantining it, since the sender is informed about the email not being delivered to the recipient’s inbox. It reestablishes email as a reliable (transactionally safe) transport, while a copy of (the rejected) spam can still be retained in a quarantine of junk folder. Halon has advocated for this approach for a long time, and it’s a prerequisite for efficient feedback and reporting mechanisms like the one I’m going to describe now.
Using CAPTCHA to handle false positives
While I believe that our default approach of rejecting (giving a 500-error) spam with an informative error message (and storing a copy in a quarantine or junk folder) is superior to a traditional quarantine, there sure is room for improvement. For example, the sender needs to contact the recipient using some other mean (alternative email or phone, which they might not have), the quarantine might consume a significant amount of disk space, and the recipient might need to bother the support staff.
We’ve developed a self-service false-positive report and release project simply called sender-fp-release to address those shortcoming. As it says on its Github page, it allows senders to report false positives directly to the recipient after completing a reCAPTCHA.
In our experience, this system is a win for everybody;
The sender doesn’t need to manually contact the recipient, only verify a CAPCHA
The recipient gets notified instantly, instead of having to browse through a junk folder
The helpdesk doesn’t need to do anything
Additionally, it saves disk space by only retaining spam for a short time (for example 1 day), unless the sender reports it. The retention time for reported email is extended (typically a week or two), giving the recipient plenty of time to release the email.
Spring is here, and so is the 3.5 release (codename sunny) of the Halon SMTP software! It includes major features such as an IDE-style script editor and object orientation, is based on FreeBSD 10.3 and comes with the latest quarterly packages. In other words, this release marks another milestone for Halon’s developer friendliness.
Noting that most of our high-profile customers uses scripting exclusively (in contrast to graphical flow chart blocks) and halonctl to configure their system, we’ve added a script editor (based on Ace and w2ui) inspired by the integrated development environment (IDE) concept. It streamlines the entire workflow by allowing you to work on multiple files, test the code you’re writing in a REPL interpreter with a pre/post-amble environment, global code search, and much more.
Another feature related to scripting is the object and this keywords, which brings object orientation to our scripting language. It’s based on closures (introduced in 3.4-r4) and is the basis for recent additions such as the MIME object. Speaking of which, the standard library’s MIME object now has send() and toString() methods which can be used to generate email (customised bounces, for example), and the DATA context’s MIME object a getBody() method. The scripting language has also received a couple of new convenience features such as destructuring assignment, null coalescing operator and include_once, which you can see examples of below. It allows you to write cleaner and clearer code, more quickly.
Google’s recent announcement that they’ll be adding encryption (TLS) and authentication (DKIM/DMARC) status icons to Gmail is a great initiative.
What they do
In addition to being displayed for received email, the encryption indicator works when composing email as well. It’s however not updated in realtime, and seems to be cached per domain. For example, a warning is displayed “@cox.net”, but not for “@coxhomesecurity.com”, despite pointing to the same MX. We haven’t been able to verify how long it takes for the warning to display; after setting up a new (non-TLS) domain one day ago and sending numerous email to it from different Gmail accounts, the warning is nowhere to be seen. We presume that the sessions of sent email are being used is material, since we didn’t record any additional probes/callouts to our test server.
I fully appreciate Google’s rationale behind displaying a red broken lock icon when TLS is absent, rather than a green lock when it is used; I just wished they addressed the shortcomings of non-authenticated TLS and took a stance on how we could move on to proper email encryption.
As I discussed in a recent blog, TLS is widely deployed for email, but virtually no-one is doing verification (checking authenticity). In the case of a web browser, TLS without verification triggers a warning (such as a red lock). It might be confusing for users that a similar symbol in Gmail indicates something different. In my opinion, there should have been at least an explaination or comment that non-authenticated (opportunistic) TLS doesn’t protect against active attacks (eavesdropping, downgrade, etc).
The main issue, of course, is that there’s currently no widely deployed mechanism for SMTP TLS name verification. This stems from the fact that email is security-agnostic (unlike the web, where you type “https://”) and that the DNS MX structure makes normal CA-based name verification ambiguous.
What they could do
The most promising technology right now for proper, verified encryption (TLS) is probably DANE, which builds on DNSSEC. It’s already implemented in Postfix (the second most popular email server) and (of course) Halon. Furthermore, many European providers and operators have either deployed, or is planning to deploy, DANE. It even serves as a cornerstone for Trusted Email Services initiative.
While its dependency on DNSSEC is likely hampering adoption, I’m encouraged by its strong community support. It feels like widespread email encryption is finally within reach!
DANE does not only challenge the certificate authority (CA) system, but also has the potential to revolutionize email security; namely making encrypted email delivery the norm. Halon Security’s mission is to bring DANE to the world of email and to help maintain data privacy and security on a global scale.
The DNS-based Authentication of Named Entities (DANE) is a proposed standard that binds X.509 certificates to DNS names using Domain Name System Security Extensions (DNSSEC).
Even today (2016), email is being transmitted unencrypted, with very few exceptions. The fact that many servers use TLS doesn’t do much more than create a false sense of security; while it prevents passive wiretapping, the opportunistic TLS mode that is being used today doesn’t protect your email transmissions from active attacks. DANE is currently the most promising piece of technology that might change all that. Despite being (primarily) a DNSSEC-based trust scheme for X.509 certificates, it also provides the means to know if a domain supports encrypted email transfer. That last part is very important; it’s what has been missing all these years.
Public-key cryptography is a fundamental component that makes the modern Internet work the way it does. One of the most widely known encryption protocols is TLS (formerly SSL), used for web browsing, instant messaging, and much more.
Email however, despite being “the go-to form of communication in the Business world” with a constantly growing user base, hasn’t changed much over the years, and is generally not encrypted.
The state of email encryption
Before we start talking about DANE, let’s first look at the differences between transport and end-to-end encryption;
Transport encryption works on domain level (an organization) and protects the email transfer. It is similar to HTTPS, used when browsing secure websites.
End-to-end encryption works on the individual level (a person) and protects the content of the email for its entire lifetime. The two most popular standards are S/MIME and PGP, but none of them have achieved widespread adoption.
While I would certainly love to see widely deployed end-to-end email encryption in the future, having transport security in place would be a very good start. When visiting your bank’s website, it’s surely encrypted. Still, when sending them an email, the transport from your email server to theirs, likely isn’t. Why is that? Since email transfer (SMTP) supports TLS, why not use a PKI such as the CA system, like we do for the web? Unfortunately, there’s no standard defining a way for the sending server to know whether to use encryption or not. For the web, you type https://, and there’s simply no equivalent for email. Until DANE.
Is the general consensus that DANE should replace CAs all together? As always there are arguments for and against. While a hierarchal system is preferable, the transfer of control and responsibility to the DNS system’s owners (governments, DNS admins) is a topic much discussion. There’s also work to improve the CA system with supplementary techniques such as keypinning and added perspective. Nevertheless, DANE stands strong to improve the security in many areas, such as email.
Most of our customers can start sending email with DANE right away, by simply changing TLS mode to dane. We recommend that you use the built-in DNS resolver Unbound with DNSSEC enabled, instead of relying on an external DNS server’s AD-bit. In the logs you can identify DANE-verified connections by
Connecting to [2001:1900:2254:206a::19:1]:25 (DNSSEC)
X.509: /OU=Domain Control Validated/OU=Gandi Standard SSL/CN=mx1.freebsd.org issued by…
DANE: validated successfully
Connection is now using TLS
If you or your customers’ domains are DNSSEC signed, you should look into receiving email with DANE as well. In theory, it’s no more difficult than publishing your SMTP servers’ certificate signatures as TLSA records in your DNS using the same name as the MX points at, but prefixed with _25._tcp, for example:
mx1.freebsd.org. IN A 220.127.116.11
_25._tcp.mx1.freebsd.org. IN TLSA 3 0 1 2B73BB905F…
We have been pioneering email security in our scriptable SMTP server for a long time; being early adopters of DMARC, DNSSEC and much more. If you’re curious, go ahead and download the software from our website.
At the center of the Halon SMTP platform is our domain-specific scripting language. As our users are well aware, the syntax is inspired by languages such as PHP (e.g. $variable) and Python (e.g. slices [0:10]). Although the implementation differs vastly, the concept of having a purpose-tailored scripting language as a form of configuration exists in other projects as well, such as Varnish.
In our pursuit of even more readable and elegant solutions, we’ve extended the scripting language to support various aspects of lambda abstraction; namely
Although many customers prefer to use the Halon SMTP Platform as-is, most hosting providers want to implement end-user interfaces in order to offer a higher degree of customer self-service. Since the Halon SMTP platform is essentially a scriptable MTA (with many features such as anti-spam, signing, and much more) with an opensource ecosystem, it makes a lot of sense to maintain the end-user interface as a GitHub project. It is designed as a boiler-plate, and our aim is that the code itself should be as simple and straight-forward as possible. It currently offers features such as indexed history, text logs, queue and quarantine management, black/whitelisting, spam settings and statistics.
In the previous weeks, we’ve added translation support using gettext, with English as the default language, and an extra translation in Swedish. If you want to maintain a translation in another language; please let us know! The language is automatically detected based on the browser’s HTTP_ACCEPT_LANGUAGE.
One of Halon’s strengths is seamless integration, something that we discussed in the previous blog entry. The same goes for the end-user interface, and we’ve done numerous integrations with common hosting platforms such as WHMCS, cPanel and Odin. The cPanel plugin is branched out into its own project, and offers session transfer for both admins and users (webmail). We recently went to Barcelona for an APS2 training hosted by Odin, and we’ve published an initial version of our Service Automation plugin. It currently offers session transfer for both admins and users, and we’re looking to extend its functionality in various ways.
We hope that you and your customers will enjoy the updated end-user experience! Do not hesitate to contact us if you want more information or help setting it up.
The Halon SMTP platform is unique in its scriptable approach to email. For those of you who still haven’t discovered the tangible business benefits of the tight integrations that this allows for, we’d like to showcase this in the video demonstration below. It surely is key to improving and streamlining your current email infrastructure.
Unlike most other anti-spam and email gateways, there’s no need to add domains and users to the Halon system. Our scriptable approach to SMTP makes it possible to avoid information duplication (which inevitably becomes out of sync) by querying (and dynamically caching) external sources. While similar to the “LDAP recipient lookup” that many email gateways support, the Halon scripting language takes you light-years further than that. Just take a look at our REST API call example that employs both short- and long-term caching and execution-time rate limits to create an incredibly robust method for external lookups via HTTP/JSON.
Fredrik has created a video that introduces those concepts, so that you can get a better apprehension for how they work seamlessly together!
Our customers use such scripts to implement dynamic routing, per-domain DKIM signing, automatic reporting and blocking of abusive users to end-user systems and much more, in order to perfectly integrate with their surrounding environment.
We’re very proud to announce the 3.4 release (codename rocky) of the Halon email gateway software! It’s based on FreeBSD 10.2 and comes with updates to virtually all components of the system; the CYREN anti-spam engines (now implemented using ctasd), OpenSSL, and many more.
Being a major release, it also comes with hundreds of new, small features. For example, it adds a SetTLS function to the pre-delivery script (yet another way of handling mail servers with broken SSL implementations), a pie chart function called q() that can be used to visualize the queue and quarantine, and zoomable line graphs on the statistics page which layout can be saved per user.
As a result of the news features in the email gateway, our logging and end-user interface has been updated to take advantage of those improvements as well. On another note, we’ve released an updated version of our cPanel plugin as a stand-alone package which adds support for the paper lantern skin.
As usual, new systems are deployed by downloading a disk image or virtual machine template, and running systems updated by simply pressing the automatic update button on the web admin’s update page.
Multiplexing the SOAP requests in the web administration is important, because typical installations have anything between 2 and 20 email gateway nodes. Some time ago we blogged about how the made the SOAP client in the end-user interface asynchronous. However, in the email gateway’s admin interface the multiplexing has been supported by the rpcmplexd (that you might have noticed in the top display) since many years, because we couldn’t use the end-user interface’s curl_multi_*-based code since it doesn’t support verifying TLS certificates based on fingerprints. This wasn’t an issue for the end-user interface, because administrators could simply import a certificate on the server. In the email gateway however, it was a showstopper since the configuration’s remote_system__* (clustering) only contains the fingerprint; not the full certificate.
Philosophically, there’s one compelling reason to get rid of rpcmplexd; its the last thing that ties the admin interface to the email gateway. Our security model in which the backend‘s SOAP API is the only way to interact with the system has enabled us to run the admin interface in a sandbox for a long time. Theoretically, one could copy the entire /var/www/htdocs directory and put it on another web server, just pointing the SOAP client at a mail gateway node instead of localhost. If it wasn’t for rpcmplexd…
Well, no more! We’ve replaced it with a future-based class which extends SoapClient. It’s a stream_socket_client-based HTTP client that supports chunks and TLS using stream_socket_enable_crypto (with fingerprint verification; that’s where it all started, remember?), which is made asynchronous (using STREAM_CLIENT_ASYNC_CONNECT, stream_set_blocking and stream_select). It’s used like a typical futures-API
and will be part of the next release. While we don’t expect any administrators to actually move the admin interface of the cluster nodes, we still wanted to share with you some of the under-the-hood changes we’re doing. The next release will also add a SetTLS function to the pre-delivery script (yet another way of handling mail servers with broken SSL implementations) and a pie chart function called q() that can be used to visualize the queue and quarantine. We’re testing the upcoming release both as a minor update to 3.3 (based on FreeBSD 10.1) and as a major (binary-diff wise) update based on FreeBSD 10.2.
Halon is a flexible security and operations platform for in-transit email. It enables companies that build and operate large-scale
email services to offer competitive features by rapid implementation, and to lower costs of maintenance through
reliable deployment and reduced complexity.