The birth of Halon’s scripting language

April 28th marks the date for Halon’s 10th anniversary and I would like to share with you the story about Halon’s scripting language, HSL. In order to understand why we created our own scripting language you have to look back at what it was intended to do, and the landscape of embeddable languages in 2007.

The Halon scripting language started out as an idea of having a dynamic configuration. We wanted people to easily be able to weight the results of different anti-spam engines (Cyren’s RPD and SpamAssassin). Hence, we came up with the idea of having a simple language with functions, ScanRPD returning the spam score from the Cyren engine, and ScanSA returning the result of SpamAssassin. The configuration could look like:

if (ScanSA() > 5 and ScanRPD() > 0) Reject();
if (ScanSA() > 3 and ScanRPD() >= 50) Reject();

In order to facilitate this, we needed a simple scripting language. At the time, the intent was not to allow any general purpose programming features. We didn’t even want loops, in order to prevent runaway programs.

Creating a domain-specific scripting language

If you’re not into programming languages, I should explain that creating a simple domain-specific scripting language is easy. There are tons of guides and it doesn’t take more than a few lines until you get simple arithmetic to work (5 + 6). The hardest and most important part of creating a language is the design, also called the syntax. You want to make it as easy as possible to read and write.

Domain-specific languages are no a new phenomena, as they have existed in a lot of different applications. I believe that custom application scripting DSLs are getting less common today, as a few selected embeddable scripting language engines are getting more traction. A few years ago you would probably pick Lua to be the embedded language of choice, while nowadays JavaScript (v8) is the language everyone knows.

Why not choose an established scripting language?

Over the years, people have asked me why we developed our own language and not used e.g. Sieve, Lua or JavaScript. Here’ why:

Sieve (rfc3028), could technically have been an alternative, but in 2007 we hadn’t heard about Sieve. It crossed our paths a few years later. Speaking against it; Sieve was created by Mirapoint, an email gateway competitor at the time. Looking back, it was probably good that we didn’t end up using Sieve. Having our own language made our own platform evolve way beyond Sieve, and what you would expect of a traditional email gateway.
Lua, it just didn’t happen and I suspect that if we would have considered Lua it would had been too large and unfamiliar as a language for our initial goal. Despite the fact that arrays starts at one ?.
JavaScript wasn’t just that common as an embeddable language and v8 wasn’t released at the time. And to be honest, in 2007 no one expected JavaScript to be where it is today.

Easy to learn and easy to build upon

Today we try to make HSL as familiar and easy to learn as possible, which is really important when you have a custom language. Everything we add or change is by the principle of least surprise. The language has copied a lot of syntax and good ideas from different languages. It may look a lot like PHP, it may even be mistaken for PHP, while other major concepts are from JavaScript and Python. Development of new language features are in many cases intentionally slow, as they needs to be well thought through. From a language designer perspective I would say that there isn’t much syntax in HSL that I don’t like. However we continuously add modern features. In the recent year or two, a lot of time has been put in to the language and it has gained features such as closures, classes and modules. They allow our language to be easily extendable so that you can build reusable modules on top of it. Our entire examples collection on GitHub can be imported as modules and a lot of them are written as classes.

One of the most innovating features of HSL is the cache statement as it allows you to cache the result of any function call based on the input arguments. Sure, the same functionality can be built in other ways, but having such a powerful tool so easy at hand in HSL makes it stand out. It gets really neat when you do network lookup queries, such as API lookups using http() or ldap_search().

cache [] http("http://api.example.com/v1/?param=$1", [], ["foo"]);

I personally really like the concept of custom languages, I think it’s important to try to evolve and challenge the concept of established languages, and by doing so we progress and learn from each other. I think every new language brings something new to the table; it can be a specific feature or the entire concept of why it was created in the first place.