Cloudflare Unveils Gen 13 Servers: A Paradigm Shift in Edge Infrastructure Driven by Hardware-Software Co-Design and AMD EPYC Turin

Cloudflare, a global leader in internet infrastructure and security, recently announced the rollout of its Gen 13 servers, marking a pivotal evolution in how its expansive network manages and processes internet traffic. This latest generation represents a significant architectural departure, moving away from a reliance on large CPU caches for rapid data access towards a highly parallelized processing model that leverages a multitude of processor cores in its new AMD-based server fleet. This strategic shift underscores a sophisticated approach to hardware-software co-design, aiming to optimize performance, enhance energy efficiency, and significantly boost capacity across Cloudflare’s global edge network.

The Imperative of Edge Performance: Cloudflare’s Role

Cloudflare operates one of the world’s largest networks, spanning over 300 cities in more than 120 countries, processing trillions of internet requests daily. Its services are fundamental to the modern internet, providing content delivery network (CDN) capabilities, DDoS mitigation, web application firewalls (WAFs), and DNS services, among others. For a company at the forefront of edge computing, where milliseconds can dictate user experience and security efficacy, continuous innovation in server architecture is not merely an advantage but a necessity. The rapid growth of internet traffic, the proliferation of complex web applications, and the persistent threat landscape demand infrastructure that is not only powerful but also incredibly agile, efficient, and scalable.

Historically, Cloudflare, like many other high-performance computing providers, optimized its infrastructure by seeking out processors with extensive CPU caches. These large caches acted as high-speed memory buffers, reducing the latency associated with fetching data from slower main memory. While effective, this approach could, at times, compensate for software that was not fully optimized to distribute workloads efficiently across multiple processing cores. As processor core counts increased, this reliance on cache became a potential bottleneck and an area ripe for optimization.

A New Architectural Blueprint: Gen 13’s Core Innovations

The Gen 13 servers are a testament to Cloudflare’s commitment to pushing the boundaries of what’s possible at the network edge. At the heart of this new architecture lies the AMD EPYC Turin 9965 processor. This choice signifies a deep collaboration with AMD and a deliberate move towards a "dense core" strategy. The EPYC Turin 9965 is a formidable piece of silicon, boasting 192 cores. This represents a substantial increase in parallel processing capability compared to previous generations and competitor offerings.

Beyond the processor, the Gen 13 specifications paint a picture of a meticulously engineered machine built for extreme demands:

• Processor: 192-core AMD EPYC Turin 9965. This processor leverages AMD’s Zen 4c architecture, designed for high core density and efficiency, making it ideal for scale-out workloads common in cloud and edge computing environments.
• Memory: 768 GB of DDR5-6400 memory. DDR5 technology offers significant improvements over its DDR4 predecessor, including higher bandwidth, improved power efficiency, and enhanced reliability. This substantial memory capacity is crucial for caching frequently accessed data and supporting the memory-intensive operations of Cloudflare’s diverse services.
• Storage: 24 TB of PCIe 5.0 NVMe storage. PCIe 5.0 doubles the bandwidth of PCIe 4.0, enabling incredibly fast data transfer rates. Coupled with NVMe (Non-Volatile Memory Express) technology, which optimizes storage access for solid-state drives, this provides ultra-low-latency storage critical for logging, data analytics, and rapidly serving content.
• Networking: Dual 100 GbE network interface cards. A 100 Gigabit Ethernet connection provides immense bandwidth, essential for a network that handles petabytes of data daily and requires seamless, high-throughput communication between servers and the broader internet.

These specifications collectively enable Gen 13 servers to handle up to twice as much traffic as the previous Gen 12, which utilized the AMD Genoa-X 9684X processor. Crucially, this performance increase is achieved while maintaining the same stringent response-time targets that Cloudflare’s services demand. The implications for infrastructure efficiency are profound: approximately 60% more capacity per rack without increasing power consumption, alongside significant increases in available memory, storage, and network bandwidth.

The Genesis of Gen 13: A Journey of Co-Design

The engineering journey behind Gen 13 was detailed in Cloudflare’s blog post, "Inside Gen 13: how we built our most powerful server yet," authored by Syona Sarma, JQ Lau, Ma Xiong, and Victor Hwang. Their insights illuminate the intricate engineering choices, from the server layout and component selection to optimizing thermal efficiency and the transition to 100 GbE networking. A core revelation from their report was the profound alignment between hardware capabilities and Cloudflare’s redesigned software stack.

The engineering team emphasized that the primary objective was to support workloads that now scale with parallelism rather than cache, thereby enabling significantly higher request capacity and superior performance-per-watt across Cloudflare’s global edge infrastructure. This marked a deliberate departure from the previous generation (Gen 12), which was powered by the AMD Genoa-X 9684X, and earlier architectures that often relied on processors with very large L3 caches to mitigate latency, particularly when parts of the software were not fully optimized for highly parallel execution.

The Software Revolution: Rust and the FL2 Stack

Central to this architectural shift was the development and deployment of Cloudflare’s redesigned Rust-based FL2 software stack. Rust, a modern systems programming language, has gained significant traction in the industry for its emphasis on memory safety, concurrency, and performance—qualities that are paramount for critical infrastructure. The FL2 stack represents a cleaner architecture, characterized by optimized memory access patterns and reduced dynamic memory allocation.

This software overhaul directly addressed the challenge posed by AMD’s newer Turin Dense CPUs. Initial testing revealed a significant hurdle: these processors, while offering high core counts, featured approximately one-third of the L3 cache compared to their predecessors. This difference initially resulted in a substantial latency increase of around 50% for Cloudflare’s workloads. This was a critical moment, forcing Cloudflare’s engineers to confront the limits of their existing software’s reliance on large caches.

Through close collaboration with AMD, Cloudflare meticulously analyzed the performance bottlenecks. This iterative process involved deep dives into CPU microarchitecture and extensive profiling of the FL2 software. The engineering teams at both companies worked in tandem, leading to a targeted rewrite and optimization of key parts of Cloudflare’s software. This intensive effort successfully eliminated the initial latency penalty, ultimately unlocking the significant performance gains promised by the Turin architecture. As the authors explained, FL2’s cleaner architecture and improved memory management meant it no longer depended on massive L3 caches in the same way its predecessor, FL1, did. This transition provided the ideal opportunity to prove that Gen 13’s throughput gains could be realized without incurring a latency penalty, validating the efficacy of their hardware-software co-design strategy.

Community Engagement and Technical Scrutiny

The announcement sparked considerable interest within the technical community, particularly on platforms like Hacker News. Readers found the architectural shift fascinating, yet many posed critical questions regarding the precise attribution of improvements—how much stemmed from the new hardware versus the extensive software rewrite. Calls for clearer benchmarks and more granular technical details were common. User gdwatson encapsulated this sentiment, commenting on the perceived lack of explanation regarding the cache issue’s resolution beyond the mention of the Rust rewrite. The community sought insights into whether specific language features of Rust contributed to the cache difference or if it simply provided a robust framework for engineers to undertake a systems-level rewrite. This reflects a broader industry desire for transparency and detailed technical deep-dives into such complex engineering feats.

Broader Implications and Future Trajectories

The introduction of Cloudflare’s Gen 13 servers carries significant implications for the future of internet infrastructure, edge computing, and sustainable technology.

Enhanced Efficiency and Sustainability: The ability to double traffic capacity while maintaining response times and increasing rack density by 60% without a corresponding increase in power consumption is a monumental achievement. This directly translates to reduced operational costs for Cloudflare and, more broadly, contributes to environmental sustainability by lowering the carbon footprint of critical internet infrastructure. As global internet usage continues its exponential growth, such advancements in energy efficiency become paramount. This aligns with broader industry trends focusing on green computing and reducing data center energy consumption.

Validation of Hardware-Software Co-Design: Cloudflare’s experience with Gen 13 serves as a powerful case study for the benefits of deep hardware-software co-design. It demonstrates that merely upgrading hardware is often insufficient; true innovation arises from a holistic approach where software is meticulously optimized to leverage the underlying hardware’s strengths. This model is likely to become increasingly prevalent as specialized hardware (e.g., AI accelerators, custom ASICs) becomes more common in data centers.

The Rise of Rust in Critical Infrastructure: The successful deployment of a Rust-based FL2 stack for such a critical component of Cloudflare’s network further solidifies Rust’s position as a leading language for high-performance, high-reliability systems programming. Its memory safety guarantees and performance characteristics make it an attractive choice for organizations building the foundational layers of the internet.

Advanced Security Features: Beyond the core architectural changes, the Gen 13 announcement also introduced PCIe encryption hardware support and improved support for thermally demanding PCIe accelerators. These features are vital for enhancing data security in transit within the server and for enabling the deployment of specialized hardware, such as AI/ML accelerators, which are increasingly critical for advanced security analytics, threat detection, and content optimization at the edge. PCIe encryption, in particular, adds another layer of defense against potential insider threats or sophisticated attacks that might target data within the server’s local pathways.

Setting Industry Precedents: Cloudflare’s bold move to re-architect its core software to match new hardware capabilities could set a new benchmark for how large-scale internet service providers approach infrastructure upgrades. It highlights that incremental improvements are no longer sufficient in a rapidly evolving digital landscape, necessitating foundational shifts in both hardware and software.

In conclusion, Cloudflare’s Gen 13 servers represent more than just a hardware refresh; they embody a profound architectural paradigm shift driven by an ambitious hardware-software co-design strategy. By embracing high-core-count AMD EPYC Turin processors and meticulously optimizing its Rust-based FL2 software stack, Cloudflare has achieved unprecedented gains in capacity, performance, and energy efficiency. This innovation not only strengthens Cloudflare’s position as a pillar of internet infrastructure but also offers valuable lessons for the broader industry on the future of high-performance, sustainable edge computing.