I’d previously written a general purpose network load balancer for fun (but not profit) which I dubbed Convey. Though the original intention wasn’t for this to be an ongoing series, its become a fun project to continue hacking on so here we are; Part 1 and Part 2 for reference.

The previous implementations of passthrough load balancing were reasonably fast, but relied heavily on the Linux kernel for packet forwarding; so still lots of room for improvement. With the increasing popularity of eBPF and XDP I decided to jump back in and try to squeeze out more throughput. In many…


In the previous post on this subject I introduced a pet project load balancer written in Rust called Convey. The load balancer is capable of a few different modes of operation such as an event driven TCP proxy, or one of a couple modes operating at Layer 3 (IP): either Direct Server Return (DSR) or Passthrough (what these mean were also described in the previous post). I talked about the architecture for DSR and Passthrough being manager-worker, with a main thread receiving packets and distributing to a pool of workers. …


Its been said cache invalidation is one of the two hard things in Computer Science. Two of the more common cache invalidation policies are the Least Recently Used (LRU) and the Least Frequently Used (LFU) policies, with LRU being the more popular of the two. At last count there were 402 Github repositories tagged as some derivative of LRU (Least Recently Used) class and a mere 36 tagged as some derivative of LFU (Least Frequently Used).

But the LFU policy is important in its own right and fits a wide variety of workloads as well. Indeed, the ever useful Squid…


Lately, a pastime for me has been learning and tinkering in Rust. As Rust is a systems programming I decided a load balancer would make a good pet project to hack on. While there are many exciting Layer 7 proxies out there, the available Layer 4 load balancers are all industrial strength and somewhat complex to get up and running. So I thought to create a more general purpose Layer 4 load balancer. These are my notes and takeaways from my load balancer side-project, Convey.

Convey

A goal of this project was to build a load balancer that easily supports Layer…


I’d previously written about motivations for writing a decentralized IP/TCP endpoint monitoring and DNS failover agent which I called Goavail. The long and short of it is the project is a cluster of lightweight monitoring agents which can be deployed to remote and disparate locations in order to either perform IP or, more recently added, TCP monitoring of specific endpoints (such as an AWS EIP). This came in handy when we decided we should have some layered availability monitoring and failover in place should something go wrong with one of our ingress procxies, an AWS zone go down or become…


Background

At Nitro we operate our cloud software offerings in AWS. However, since we strive to be vendor neutral we run our ingress traffic through Haproxy instances with Keepalived instead of the AWS ELBs. You can read more about Keepalived elsewhere, but the short of it is Keepalived uses the VRRP protocol to detect when HAProxy is down and fails over an AWS EIP with minimal downtime. We additionally leverage Cloudflare’s CDN-like proxying service. In this way we can DNS round robin between multiple Haproxy instances and be assured the EIP will be associated to a live process.

Haproxy with Keepalived

This is all…


In Nitro Engineering we have adopted an All-Ops, Immutable Systems ethos. Of course, adopting the right tools is critical to this culture and products such as Docker and Ansible have served us well in engineering new systems and subsequently pushing features ever faster. However, a challenge remains in how to maintain the legacy systems we’ve inherited which don’t necessarily lend themselves to this culture. We run a lean DevOps team here at Nitro, but this is an added pressure on the typical DevOps breadth of responsibilities. …

Ben Parli

Learner, Doer, Occasional Provocateur

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store