Cloudflare has reclaimed more than 100 terabytes of RAM across its global infrastructure after engineers discovered that one of its routing systems was storing far more hash data than it actually needed.
The company detailed how changes to its Pingora Backend Router, or PBR, reduced the memory consumed by the consistent-hashing structures used to route cacheable traffic.
Cloudflare said small changes to a single algorithm reclaimed more than 100TB of RAM globally, on top of another 100TB previously recovered through separate DNS-cache optimizations.
Tom’s Hardware independently reported that the biggest change involved cutting the number of server hashes by roughly 90%, from around 100,000 to 10,000 per server.
Consistent hashing was consuming gigabytes of memory
Cloudflare uses Pingora, its Rust-based networking framework, across infrastructure that handles traffic distributed among large numbers of servers.
Its pingora-ketama library implements consistent hashing, which helps ensure that the same content is repeatedly routed toward the same cache servers even as infrastructure changes.
But Cloudflare’s routing requirements resulted in many separate hash rings.
In some processes, those structures were consuming as much as 6GB of memory before engineers investigated whether every stored hash point was actually necessary.
The existing configuration could produce roughly 100,000 hash points for a typical server.
More points improved load distribution—but only up to a point.
The final 90,000 hashes were buying very little accuracy
Cloudflare engineers derived a mathematical model for how the number of hashes affected load-balancing error.
The result showed diminishing returns.
At approximately 100,000 hashes per server, the final 90,000 hashes improved load-distribution accuracy by only about 0.7%.
That allowed engineers to reduce the hash count dramatically without materially changing routing behaviour.
They also attacked the memory layout itself.
Each stored point previously used two 32-bit values: one for the hash and one for the server index. Because no Cloudflare data centre needed more than roughly 65,000 server indexes, the server field could be reduced to 16 bits.
Cloudflare consequently reduced each stored hash point from eight bytes to six, cutting another 25% from the structure’s memory requirement.
Deployment was harder than changing the algorithm
Replacing a consistent-hashing configuration in production introduces another risk: suddenly mapping cached objects to different servers could cause cache misses and flood origin systems with requests.
Cloudflare therefore ran the old and new hash rings simultaneously and migrated traffic gradually by data centre, while monitoring memory, connection errors, cache behaviour and origin traffic.
The optimised implementation has been incorporated into the newer version of pingora-ketama.
Infrastructure efficiency becomes a product at scale
The story illustrates why infrastructure optimisation looks different at hyperscale.
Saving two bytes from a data structure would be irrelevant in most applications. Removing tens of thousands of unnecessary entries might save only megabytes on a small deployment.
Across thousands of servers and enormous routing tables, those same changes become 100 terabytes of physical memory.
That affects more than technical elegance. RAM consumes money, rack capacity and electricity. Memory recovered from existing machines can be used for additional workloads without immediately purchasing more hardware.
For infrastructure engineers, Cloudflare’s experiment is therefore a reminder that scaling does not always require adding machines.
Sometimes one of the largest capacity upgrades available is hiding inside a data structure everyone stopped questioning years ago.
