Browse

Tag: releases

Halon 4.7 “ahoy” and 4.8 “truly” with live debugging and HELO script

Halon 4.0 introduced a feature we call “live staging” where you can deploy multiple running configurations at the same time, with per-connection conditions. It allows you to reliably rollout changes or new features to a production system for only a few testing IPs, or a select percentage of the traffic. With Halon 4.7, we proudly present “live debugging” using which you can add logpoints to your scripts. It enables you to inspect the full context of SMTP transactions in real-time, using the live staging conditions as connection selector.

Those points are added directly to the Monaco-based IDE, and results are inspected on a per-connection basis. You can create multiple points, triggered by multiple messages, and jump back and forth between them.

We’ve also added a HELO/EHLO phase script, support for ARC in DKIMSign() and a full implementation of draft 18 on Github, EdDSA (ed25519) and a native boolean type with corresponding strict comparison operator. The standard library have many new functions such as rsa_sign() and verify, idna_encode() and decode, aes_encrypt() and decrypt.

We hope that the live debugging will come handy! Please see the changelog on Github for a full list of improvements and changes, or get in touch with us if you want more detailed information.

Halon 4.6 “curry” with outbound anti-spam

You probably know from before that Halon’s scriptable SMTP server enable email providers to avoid blacklisting and increase deliverability. The 4.6 release, “curry”, contains Cyren’s outbound anti-spam (OAS). In combination with our cluster-synchronised rate limit function, it provides incredibly effective and accurate abuse prevention. Just like their Cyren’s inbound anti-spam, OAS uses a hash-sharing technology called recurrent pattern detection (RPD) that identifies outbreak patterns. It’s designed to detect spam from internal sources rather than external, and doesn’t report/contribute any signatures since it could blacklist your own infrastructure.

With the flexibility of scripting you can determine customer/sender identities accurately even in mixed traffic. This is used as identifier for rate limits based on classifiers such as Cyren’s OAS, delivery failure rate, queue size, etc. By using IP source hashing and alternative IPs for suspicious traffic, deferring obvious abuse and controlling connection concurrency, you can achieve high deliverability with minimal administration.

The 4.6 release comes with many additional features and improvements. It adds SNI support to the TLS functions. The Monaco-based code editor now have additional code completion, built-in documentation, tabs, and a mini-map.

For more information on the release, see the full changelog on GitHub. If you want to try Cyren’s outbound anti-spam, contact our sales team.

Halon 4.6 “funny” supporting our SMTP LANG extension

In the beginning, everything was ASCII and English. Since then, we’ve seen Unicode (international character sets) and IDN (international domains names) become widely adopted. Last year we implemented SMTPUTF8 that enables international mailboxes.

So why not support other languages in text-based protocols? We give to you “The SMTP Service Extension for Protocol Internationalization” RFC draft, introducing the EHLO keyword LANG. It will be the first SMTP software to support our to-be submitted RFC draft. Initially it will support Swedish, Spanish and Australian, and will default to Swedish when talking to supported systems.

EHLO example.com
250-LANG SE ES AU
LANG SE
250 Ok
BREV FRÅN:<>
250 Tack
BREV TILL:<hå[email protected]än.se>
250 Tack
INNEHÅLL
Subject: asdf

Hej!
.
250 Togs emot
HEJDÅ
250 Vi ses!

If you made it this far, April fool! We will publish information on the upcoming 4.6 release some time after the 1st of April.

Happy easter!

Halon 4.5 – gettin’ certy with it

The main focus in Halon 4.5 release is TLS, hence the name “certy”. Check out the the new features and functions and try them out. Also, the knowledge base is growing with a lot of good how-to’s to help you around.

TLS information has been made accessible in the Halon Platform scripting language, both on the receiving and sending side. Support for X.509 client certificates has been added, allowing you to both verify the sender identity in the SMTP server, as well as identify yourself when sending email through an SMTP client.

Experiment: we configured a busy email system to ask for a client certificate for all inbound connections, and found that approximate 5% of all traffic provides a client identity. Most of the traffic is from Gmail and Office356. We did not collect the percentage of domains, which we leave as an exercise for you.

$peercert = GetTLS();
$haspeercert = isset($peercert["peer_cert"]);
stat("peer-cert", ["yes" => $haspeercert, "no" => !$haspeercert]);

How to enable this feature and start authenticating clients was documented as KB article.

Implementation and facilitation of TLS reporting (tlsrpt) has begun. It is a new standard for reporting TLS failures, mainly focused on MTA-STS and DANE.

The TLSSocket() class now have a getpeercert() function and the ability to specify a client certificate. Now you see why we called it” certy”?

Support for custom SASL authentication mechanism has been added. This allows you to build authentication schemes such as OTP, OAUTHBEARER or CRAM-MD5, but also EXTERNAL to facilitate the client certificate features. The procedure is documented in our knowledge base along with two sample implementations.

If you haven’t found our knowledge base before, the KB is a place to find how-to’s. The dev team is expanding it as fast as we can, adding topics that customers have asked about.

Finally, I want to highlight the big effort we’ve done to simplify, modernize and overall improve the web administration. This is an ongoing project, and something that we’re paying a lot of attention to. We want to thank, and congratulate, the Bootstrap team for providing such a awesome framework. We managed to get the Bootstrap 4.0 release in, with just a few days of work.

You can read the full changelog on our GitHub of all the other features big and small.

Halon 4.4 “lofty” packed with small improvements

The 4.4 release “lofty” is all about fixing bugs, boost existing features, and improve performance and memory management in the Halon script engine. And like macOS “High Sierra”, it’s fully baked.

The unusually long changelog contains many small improvements. We’ve given the pre/post-delivery script a slight overhaul. It’s now possible to tailor the bounce behaviour via the the SetDNS() function. Additionally, we’ve added $action and $context, as well as functions to set MAIL/RCPT parameters. Finally, the SetSouceIP() enables you to choose an IPv4 and IPv6 address pair, which is a great when you want to provide customers with a private IPv4 and IPv6 or if you want to use diverse address pools.

The improved “Listen on” directive on the Server > SMTP listener page enables more fine-grained control over listen ports and IPs; such as listening on different ports for different IPs.

Quirks and fun trivia
  • We recently revised our LDAP implementation, and realised that our own syntax and mechanism for failover between hosts is rather superfluous, since OpenLDAP supports that natively. Consequently, we adopted the standard LDAP URI’s in our configuration, and existing configurations will be automatically migrated.
  • While we support the PROXY protocol (v1) that passes client source IP information from load balancers, we thought it was mostly as HAProxy thing. Apparently, it’s used by many other load balancers such as Amazon ELB, Citrix Netscaler, and F5 BIG-IP. Most of them implements the version 1 (which is human readable), but there is a second version of the protocol that’s binary-packed, and have a quite smart feature: its magic string (protocol identification) is \x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A which translates into literal "\r\n\r\nQUIT\r\n", a string chosen specifically to case an error and disconnect against servers not supporting this protocol. Clever!
  • If you have a IPv6 only datacenter, but still want to process IPv4 clients, you can do so with a SIIT-DC gateway which uses IPv4-mapped-IPv6 addresses. In Halon, you can use SIIT-DC while still performing IPv4 reputation (such as DNSBL), by extracting and setting the IPv4 address in the CONNECT script. If that doesn’t make the point that we’re very scriptable, then what does?

Image from Tore Anderson’s SIIT-DC presentation

If you ever had problems signing in to a Halon using Firefox, it can be because a recent change in how “secure cookies” are handled. When signing in over HTTPS, we set the secure cookie flag, which forbids the cookie to be send over a unencrypted HTTP connection to the same host. That is all great, but if you then try to sign in over HTTP (for whatever reason) Firefox will not be able to login because there is already a cookie for that domain with the secure flag and it cannot be replaced, nor accessed. We addressed this by using different cookie names for HTTP and HTTPS. Regardless of this fix, you should not use HTTP when administering your Halon hosts.

Time-of-click protection against ransomware, malware and phishing

Time-of-click protection adds an extra layer of security to protect email users from accessing malicious content. Attacks including malware, ransomware and phishing are becoming more common and more sophisticated with every day, along with users keeping more sensitive information. 

With an additional time-of-click protection, Halon will classify links in email every time it’s clicked, before allowing or denying the user to visit it. This means that if the scammer waits two minutes or two months with infecting the site, the user will still be protected when he or she chooses to click the link. It’s the extra layer of security that won’t allow you to visit infected websites by way of a link in an email protected by Halon.

Features

  • On-premise or hosted cloud
  • Front/backend architecture for high availability
  • Multi-tenant with companies and users
  • Supports branding
  • Multiple detection engines
    • CYREN
    • Sophos
    • Google Safe Browsing
    • ISITPHISHING.org
    • Spamhaus
    • SURBL
    • URIBL
    • PhishTank
  • Optional click history
  • Black/whitelisting

Time-of-click protection is an add-on to Halon SMTP software, and we recommend that you extend your license to include it. Pricing is set per user, with volume-based discount. If you are already a Halon customer, contact your sales representative, or send us an email for your quote.

Meet “classy” and “cody”, Halon 4.2 and 4.3

We have done two new releases of Halon since last time we updated the blog with release matters. In Halon 4.1 “teamy”, released just before this summer, we introduced modules. A month later we followed up with 4.2 “classy” that added proper object orientation to the language (which works great in combination with modules). It spawned a few rewrites of our script examples (modules) to reflect this awesomeness. We initially added instance and class methods and variables (static), and in 4.3 “cody” we added the private keyword to functions and variables as well.

class HelloWorld
{
	private $name = "Dr Who?";
	constructor($name)
	{
		$this->name = $name;
	}
	function sayHello()
	{
		return "Hello ".$this->name";
	}
	static function ...()
	{
		...
	}
}

We’ve created a lot of modules and script examples. Some of those, such as the PostgreSQL and MongoDB modules, rely heavily on byte packed data structures. In order to better support those, we’ve added built-in functions such as pack() and unpack(). Upcoming modules and rewrites will also benefit from the new TLSSocket() class.

Here are some new additions to our module collection:

Other notable features from the changelog includes

  • FreeBSD 11.1 and new quarterly packages
  • sha2 hash functions
  • Added status and NDR codes to Reject, Defer and Deliver functions
  • SetTLS support CA name verification
  • DLP engine now support file hashes of SHA2-256 and SHA2-512
  • Added $sourceip variable to post-delivery script to easily determine which IP address that was used to send the mail
Geek out corner

One major change that only we can see and fully appreciate is the (both automated and manual) code migration to C++11 (and forward), using the truly awesome clang-tidy tool.

On another note; while we researched pack and unpack implementations by looking at other languages’ documentation (such as PHP, Perl and Python), we found a bug in PHP, which was fixed in 7.2, and backported to 7.1.9. The overall consensus of syntax and conventions amongst languages regarding how pack and unpack should work seems to reflect and mimic Perl.

Some scripting languages like JavaScript and HSL has the notion of class constructors but no destructors. The HSL memory model uses references counted automatic garbage collection to determine when objects should be removed.

“In a language with an automatic garbage collection mechanism, it would be difficult to deterministically ensure the invocation of a destructor, and hence these languages are generally considered unsuitable for RAII [Resource Acquisition Is Initialization]” – Wikipedia on destructors

MongoDB does unlike many other databases use little endian and not big endian (network byte order) in its wire protocol. This will let you send and receive data structures in native machine endian (for most people) since both x86 and amd64 use this convention. I highly recommend reading up on the fun historic trivia about endianness.

Want more in-depth info on the new releases? Get in touch with the support team.

Halon 4.1 “teamy” with scripting modules

We’ve shipped the 4.1 “teamy” release, and we’re happy to see that the majority of our users have already updated. The 4.0 release brought several major changes, and 4.1 includes many improvements to it. The web administration in general, and the script editor in particular, have received lots of attention. The new live staging functionality that was introduced in 4.0 now extends to queue (pre- and post-delivery) scripts as well, using the _stageid metadata field. You can employ even more aggressive caching now that the API includes a hslCacheClear() function, and we’re opened up a wide range of possibilities via the new setBody function in the DATA script’s MIME class, such as URL rewriting.

The most anticipated addition however, is a new language feature often referred to as modules. The Halon scripting language is highly email-centric. We strive towards keeping it as simple as possible, which is a great recipe for stability and high, predictable performance. At the same time, we’re designing it after the principle of least astonishment, and consequently it shares many characteristics with established languages. Having researched both namespaces (found in languages such as C++ and PHP) and modules (found in for example Python and Perl), we decided that modules was the best fit for our language and its users. It’s great when working in larger teams, since it introduces file-wide symbol (variable and function) scoping. Modules are regular script files, which becomes modules the moment you import them. It’s similar to include, except an imported file has its own scope. You explicitly choose which symbols to “import” to the parent file using the syntax

import { Foo as X, $bar as $y } from "test";

Variables are imported by reference, hence all changes to the variable in the module will be reflected by the imported variable. As modules have their own global scope, variables referenced in a module’s function by global or closure points at the module’s scope, as per the example below

$x = "hello";
function foo($bar) {
    global $x;
    echo $x;
    $bar();
}

imported by

import { foo } from "test";
$x = "world";
echo foo(function() closure ($x) { echo $x; });

which will echo

hello
world

We hope that the modules concept will be helpful in both smaller and larger projects, and we promise that much more is to come as Halon 4.2 “classy” hits the servers.

Live staging and SMTPUTF8 coming up in Halon 4.0

Halon 4.0 a.k.a. “forty” is around the corner, and it has a strong focus on developer friendliness and deployment. Let’s take a peek on what’s about to happen!

Most of the work has gone into the SMTP server. Most notably, it now supports running two different configurations, complete with compiled scripts, in parallel. We refer to it as “live staging”, since it allows you to try out new code on a production host, for only some select traffic, based on conditions, such as IP address. We’ve also introduced a new persistent connection $context variable to all the SMTP server’s scripts, including the new MAIL FROM script, which can be used to pass data between SMTP stages or even RSETs, like

$context[$messageid][$recipient] = lookup($recipient, ...

since $messageid is regenerated with each RSET. Another major feature is SMTPUTF8, which allows for international (and emoji!) email addresses.

The scripting language itself has a new resource value type, used by the Socket() networking class which, in turn, is used to implement various extensions such as a memcached client. Another new scripting features is bitwise operators, which you can see in action by viewing the TOTP example’s code.

In addition to the live staging, management is significantly more distinct and reassuring thanks to a checkout/commit pattern, and a more expressive configuration format. As you begin to make changes, already tested using the live staging, as larger commits with descriptive messages, browsing the configuration revision history will become much more enjoyable. It also supports larger teams much better.

Halon 4.0 user interface

As usual, it’s based on the latest FreeBSD version; 11.0 in this case. It features LLVM’s LLDB, Receive Side Scaling (RSS) in some Intel network drivers, unmapped IO in Xen and VirtIO, Amazon AWS SR-IOV networking, Tx/Rx multiqueue in VMware VMXNET3, and much more.

Check out the changelog for more detailed information, and try out the new web interface at demo.halon.io!

This year, wish lists will be sent to Santa via email

Ever got an email from Santa Claus? Me neither, but in this day and age I’m assuming it’s just a matter of time. And when I do, I want to see his face in the ”From”-field. Yes, I said face, not name.

When writing an email in Gmail, my quickly typed 😉 easily transforms into a graphic smiley. Reinforcing my attempt to create a feeling with the reader. If I’m typing on my smartphone I might also include some other emojis like the ❤️. And then I send it to, for example, [email protected]. Until now, when I actually can send it to, or from, 🎅@northpole.com.

The character encoding known as UTF-8 makes it possible to include emojis in email addresses, and all possible characters, such as for example the Swedish å, ä, ö. The latter is certainly more useful than the first, but to any marketer (or jokester) it will most definitely be a fun detail to include when sending someone an email. In the upcoming Halon 4.0 we will include support for SMTPUTF-8 (RFC6531). But what about…

Typing?
Typing an emoji on your smartphone is obviously easy but what if you are on desktop? A Mac-user will press ctrl+cmd+space, but a simple Chrome plugin or even copy/paste from an emoji library website will also do!

Conversion to letters?
Regular email addresses are almost always converted to lower case, to make filtering and routing work. For Halon 4.0 we will convert UTF-8 to upper- and lowercase alternatives. Because there is no guarantee that a next-hop SMTP server will support the SMTPUTF8 extension, use of the SMTPUTF8 extension always carries a risk of transmission failure.

Halon 4.0 will be keeping the team busy during the holidays, until the release, sign up for the newsletter (in the website footer) and make sure you don’t miss the release or any interesting articles on the world of email. And from all of us, to all of you, merry Christmas and a Happy New year!