Pingora, is a new HTTP proxy created by cloudflare using Rust
Cloudflare unveiled recently who is migrating your content delivery network to use the proxy «pingora», which stands out for being written in Rust. The new proxy replaced NGINX server-based configuration with Lua scripts and handles over a billion requests per day.
It is noted that the transition to the specialized proxy «Pingora» allowed not only to implement new functions and increase security due to safe work with memory, but also led to a significant increase in performance and resource savings.
A Pingora-based solution does not require the use of Lua and uses Cloudflare's load-optimized architecture, consuming 70% less CPU resources and 67% less memory while processing the same amount of traffic.
For a long time, a traffic proxy system between users and end servers based on NGINX scripts and Lua met Cloudflare's needs, but as the network grew and its complexity increased, a universal solution was not enough, both in terms of performance and limitations in extensibility and implementation of new opportunities for customers.
In particular, there were difficulties in adding functionality beyond a simple gateway and load balancer. For example, in the event that the server was unable to process a request, it was necessary to redirect the request to another server, providing it with a different set of HTTP headers.
Instead of an architecture with requests divided into separate work processes (workers), Pingora uses a multi-threading model, which in Cloudflare's usage scenarios (high concentration of traffic from different sites with a large statistical shift) showed a more efficient distribution of resources between CPU cores.
In particular, binding unbalanced requests to processes in nginx led to an unbalanced load on CPU cores, as a result of which resource-intensive requests and I/O blocking slowed down the processing of other requests.
Also, binding the connection pool to controller processes did not allow reuse of already established connections from other controller processes, which reduces efficiency when there are a large number of controller processes.
The introduction of Pingora made it possible to reduce the number of operations by 160 times establishing new connections and increase the proportion of reused requests from 87,1% to 99,92%. In addition to reduced reconnections and more efficient use of CPU cores, the performance improvement of the new proxy was mainly due to the removal of slow Lua drivers used with nginx.
The Rust language was chosen because it allows high performance combined with the availability of memory-safe tools. It is mentioned that despite highly qualified Cloudflare engineers and code review, programs written in the C language could not avoid bugs that lead to memory problems (for example, a vulnerability in the HTML parser).
As for the new code, we talked about crash analysis cases in Pingora, which turned out to be caused not by problems in the application, but by a bug in the Linux kernel and hardware failures.
Additionally, it can be noted that Linus Torvalds commented on the inclusion of support for the Rust language in the Linux kernel, expressed at the Open-Source Summit Europe conference that takes place these days. Kernel 6.0 didn't include patches for developing device drivers in the Rust language, but according to Linus, they're likely to be accepted in kernel 6.1, you're not going to take advantage of the integration.
As a reason to add support for Rust, in addition to a positive security impact, Linus is also taking the opportunity to increase interest in working on the kernel from new entrants, which is important in the context of aging veterans.
Finally if you are interested in knowing more about it, you can check the details in the following link.