At Halon, every feature we build is driven by a commitment to solving real customer challenges with precision and innovation. Delivery Orchestrator is the latest step in this journey, created to intelligently manage traffic distribution across instances in a way that adapts seamlessly to real-time conditions. 

Our journey with Delivery Orchestrator began a few years back when one of our clients, a high-volume email sender, faced a common but persistent problem. In a clustered setup with multiple instances, delivering to destinations with strict connection limits posed a unique challenge. Traditional rate limits and concurrency settings often caused certain servers to become overloaded while others remained underutilized. This imbalance created inefficiencies in managing email traffic across the cluster. Initially, we implemented a distribution method by dividing email delivery limits equally across instances, which seamlessly integrates with Kubernetes for dynamic recalculation as the number of instances are scaled up or down.

We recognized that static distribution is highly effective in many scenarios, providing a straightforward and reliable way to allocate resources across instances. However, depending on factors such as traffic distribution and complexity of traffic shaping demands, there are scenarios when the limits of a purely static approach become apparent.

For email operations dealing with variable loads and very strict destination limits, a more dynamic solution was needed—one that could adapt in real-time, respond to fluctuating traffic, and maximize resource efficiency. From this need, the concept of Delivery Orchestrator was born: a feature that goes beyond simple load balancing to orchestrate traffic intelligently, ensuring that every instance in the cluster contributes effectively under demanding conditions. Compared to static division, Delivery Orchestrator prioritizes instances with more messages to send, achieving faster throughput, avoiding bottlenecks and optimizing resource utilization.

Building Delivery Orchestrator

Creating Delivery Orchestrator required us to approach email traffic management in a new way. Unlike a traditional load balancer, Delivery Orchestrator needed to take multiple factors into account:

• Concurrency and rate limits: We wanted to ensure that traffic limits were adhered to dynamically, so each instance received the resources it needed without overwhelming any single node.
• Real-time traffic management: Our clients operate in high-scale, cloud environments like Kubernetes, where new instances spin up or down frequently. Delivery Orchestrator would need to adjust limits as these changes occurred, without any manual input.
• Fairness and efficiency: Balancing fairness with efficiency was critical. If one instance had a heavier queue, Delivery Orchestrator would need to grant it more resources temporarily while keeping an even distribution of total traffic over time.

Our engineering team evaluated different architectures to find the right balance of simplicity, speed, and resilience. In the end, we found a solution that gave us the performance and low-latency responses needed to handle real-time adjustments across all instances in the cluster.  Delivery Orchestrator offers a level of traffic management that’s both flexible and robust, enabling clients to handle surges and large delivery volumes with ease.

How Delivery Orchestrator improves email delivery

Efficient email delivery is the backbone of successful email infrastructure. Here’s how Delivery Orchestrator helps clients achieve that:

• Higher throughput: By dynamically adjusting traffic distribution, Delivery Orchestrator enables each node to operate at peak efficiency, resulting in faster overall throughput.
• Resource optimization: Rather than idle resources on low-load instances, Delivery Orchestrator ensures every available connection is maximized, making the most of the infrastructure investment.
• Reduced delays: Delivery Orchestrator minimizes delays by allowing nodes with higher queues to process more connections, reducing latency in email delivery.
• Seamless scaling: The feature is cloud-native, making it perfect for environments where demand fluctuates, ensuring that the infrastructure can scale smoothly without manual intervention.

Continuing to listen, improve, and adapt

Delivery Orchestrator reflects Halon’s engineering philosophy: we don’t just check off a feature and move on. Every improvement, no matter how small, is part of a larger commitment to help our customers succeed. From the first iterations to the real-time orchestration we offer today, Delivery Orchestrator is a testament to Halon’s dedication to evolving alongside our customers’ needs to provide robust, scalable, and resilient email infrastructure that fits their needs.

Looking ahead, we’ll continue to refine Delivery Orchestrator, exploring new ways to make it even more responsive and powerful for large-scale email senders. And, as always, our clients’ feedback will be at the heart of every enhancement we make.