We are just as enthusiastic about the prospect of tri-hybrid systems in the data center as we were more than a decade ago. We firmly believe that CPUs, GPUs, and FPGAs all have a place in complex and high-performance systems because these devices have their own strengths and because elements of most application workflows are serial and parallel processing that is either relatively static or constantly changing.

01 Introduction of CPU

Programmability of CPUs and GPUs is critical when code is constantly changing, and many applications are written in C, C++, Java, Python, Fortran, and a few other languages, either running close to the hardware or in a virtual machine that abstracts the underlying hardware and interprets it on the fly. CPUs are meant for serial work, and getting answers quickly usually means running on fast cores, and for algorithms that are numerically intensive and do a lot of multiplications and additions, you can offload that to GPUs, which typically have an order of magnitude more parallel processing power than CPUs.

FPGAs fit into this trifecta of hybrid systems, where high performance in compute or data processing is important, but the algorithms of the workloads don’t change much—maybe every few years, maybe a few times a year—rather than every day or week as happens in many industries. An FPGA can do some work faster than a CPU, but it’s never as energy efficient and it’s certainly not cheap, but it can offer better price/performance and works much better for certain types of dataflow work. (An FPGA is a malleable dataflow engine, even if it increasingly has hard-coded components like Arm cores, DSP blocks, and SerDes communications.

Perhaps more importantly, sometimes you need more performance than an algorithm written in C++ or Java for a CPU can provide, but you don’t need such high-volume silicon that it makes sense to make a fully custom ASIC. This happens all the time in telecom and networking systems. This is different from prototyping on FPGAs (usually groups of FPGAs), which is used to create ASIC models so that they can be tested, which is another important use of FPGAs. We think of this as ASIC prototyping, which we mentioned above as being used in production inside a switch or some other device; it’s not prototyping, but rather a middle ground between a general-purpose CPU algorithm and a fully custom ASIC.

02 History of Altera

When Intel bought Altera for $16.7 billion in June 2015, one of the ideas was that what we now call a DPU was that application/device that needed speed (it's an arbitrary word) that you paid extra for, and it would run better on an FPGA than a general-purpose CPU. Some thought that a third of hyperscale and cloud servers would have these on-the-bus, in-system FPGA accelerators. Intel calculated server shipments a decade from now and figured out that all of these functions would be moving off CPUs, and it would be better to have a device to attract these customers. Ding! Altera shareholders cash out, Intel buys FPGAs.

Part of the idea was also that many of the HPC and AI algorithms that were offloaded to GPUs could be moved to FPGAs. At the time, AMD didn't really offer a data center GPU product that competed with Nvidia's product line, but it was better than Intel, whose GPUs were only used in laptops and entry-level server chips. Still, given AMD's desire for more expertise in accelerated computing and its FOMO on the FPGA wave, it didn't surprise us, you, or Wall Street that AMD wanted to acquire rival FPGA maker Xilinx in October 2020, a deal that closed in February 2022 for nearly $49 billion.

Since then, Intel's data center GPU business has languished, while AMD has finally gained ground with its competitive hardware. Both companies believed that FPGAs had a place in the world and in the data center, but perhaps cooler heads prevailed.

Sandra Rivera, who previously ran Intel's data center and AI business units and has been named CEO of the independently reorganized Altera, hosted Altera's Innovator Day this week, and during a pre-event briefing, we had a chance to talk about FPGAs in the data center and the opportunities ahead—especially given that hyperscalers and cloud builders, by and large, aren't using FPGAs for the central compute engines of accelerators and DPUs that they say they are. We'll quote Rivera in full because the answer is complicated because the question is:

"If you look at the thesis for Intel to acquire Altera, it was very much built around the data center and enterprise opportunity, both in cloud and enterprise infrastructure, and communications service provider networks, both in radio access networks and wireless networks, and centralized core, which is a lot of data centers. I would say that we actually did very well in terms of what we thought we could pursue and what we could win from the perspective of the emergence of IPU slots, which is something we may not have seen in 2015, but we've seen more and more over time. More and more cloud service providers want to offload so they can sell cores on the CPU. And then they want to offload anything that has to do with storage acceleration or platform overhead management. We actually won every socket available for FPGAs in the market. There are a couple of sockets dedicated to ASICs. Of course, you're familiar with AWS Nitro, but Intel actually won another big win with Google and the "Mount Evans" platform. But every other IPU slot, Altera, then PSG, and now Altera, has won. So we've done very, very well there."

03 Different views of Altera

"We also did very well from a communications service provider perspective as we went through this whole 5G transition, and we won a lot of bid platforms from what we call TEMs (telecommunications equipment manufacturers). We also won a lot of platforms, not only in Europe and the U.S., but also in China. Now, some of them are obviously restricted by export controls, but we won a lot of platforms."

"We really didn't envision that we could co-package FPGAs more with CPUs. For example, I don't think we realized at the time how much you would constrain one device or the other when customers really wanted to use both devices to their full potential. So that value proposition never materialized and we abandoned that strategy in the first few years. And then the other area, now that we're independent, we're driving the FPGA strategy as an FPGA company, which is very different for us, is the mid-range, lower-end, power-constrained, cost-constrained environments and segments that are not as important to us to drive Intel's CPU-first strategy."

"As I said, we didn't invest in building out the distribution network and all that capability, and of course, now we're completely focused on reference kits and expanding the reach and scale of the portfolio through our partners. And that's because, from an Intel strategy perspective, it’s not as important as the top 25 that make up the majority of Intel's revenue. "So we obviously learned some lessons along the way. We made some adjustments. We did well in the areas where we were focused. We didn't do as well in the areas where we didn't invest as much time and effort. You know, that's what we're going to change now because that's all we do, we work on all use cases, applications, market segments and capabilities in our portfolio."

This is a history and explanation that we won't dispute, but we further pressed for a more specific answer on the future of FPGAs in the data center and quipped that the FPGA opportunity in the data center is smaller today than we thought a decade ago, but not less than it has been in the past few years. To this, Rivera responded:

"It's a huge opportunity. In fact, in this part of the market, it's so important to keep the focus on the data center, and certainly with the cloud service providers and communications service providers, because that's where the technology transformation happens. So that's where you get the new memory technology. That’s where PCI-Express Gen 6.0 and 7.0 are coming out first. That's where AI is driving so much innovation. That's where you need more networking and transceiver capabilities. That's where you need interconnects like UCI-Express and CXL capabilities, and in particular, we've had a lot of wins recently in test and measurement. They need those first because when these devices come out of the product development organization, you need to be able to test them. So the test and measurement industry is typically ahead of the curve."

"But you'll also see some of those IPU sockets or some of those AI NICs eventually move to ASICs because there are so many volumes that you can cross that crossover point. Given the development costs of ASICs, especially in new process technologies and the complexity of packaging and process nodes, that may be true for a few customers and companies that have that capability, but there will always be new device refreshes that move to ASICs. What are the other opportunities that we're pursuing in FPGAs? It's always where you do experiments and pilots, in many cases, for multi-year deployments, and in other cases it's more ad hoc, depending on when you can make a business case to move to ASICs."

The task ahead for Rivera is daunting, but someone has to do it. It's important to remember that Altera and Xilinx have a long history of competing against each other in the programmable logic device space. Here's the roadmap for both companies:

Business roadmaps are driven by product roadmaps, with Altera planning to have its Agilex 3 FPGAs in production by mid-2025 to complement the Agilex 3, Agilex 7, and Agilex 9 devices already available today.

The Agilex 3 is a small FPGA with a pair of Arm Cortex-A55 cores and between 25,000 and 135,000 lookup tables (LUTs) of programmable logic capacity. The Agilex 3 may end up in some data center devices, but they're really aimed at edge, vision processing, and factory automation workloads that need some AI acceleration and auxiliary compute. Importantly, the Agilex 3 devices offer 1.8 times the performance of the relatively older Cyclone V devices they replace.

That may not be exciting to the data center crowd, but distributors of FPGA boards account for 80% of Altera's unit sales and 50% of revenue, and this product is aimed squarely at them as the reorganized Altera gets back in touch with the channel, so it's intended to get Altera's business back on track. Just as PC chips paid for Intel's move into the data center, Agilex 3 can generate revenue and profits that will allow Altera to invest in FPGA devices that data center customers may choose for workloads (or portions of them) that are not well suited to CPUs and GPUs.

We have long believed that Intel would aggressively push FPGAs into the data center and become the creator and seller of pre-configured FPGA "software"that would be sold on its PACs (short for PCI-Express Accelerator Cards). Rivera has blocked Altera from making and selling cards and left that to its partner channel.

The advantage of FPGAs and the factor that is holding back a mass migration to ASICs is the increasing development cost per process node, which she cites as an interesting example from Taiwan Semiconductor Manufacturing Company: $40 million for 28nm devices, $100 million for 14nm devices, $300-400 million for N7 devices, $600 million for N3 devices, and $800 million for N2 devices. This means that the volume crossover that can afford to develop custom ASICs is absolutely huge. While Rivera acknowledged that FPGAs won't replace GPUs for large-scale AI training—and nobody expects that—FPGAs have big opportunities in pre- and post-processing of AI training, AI inference, 6G wireless platforms, and security processing.

This may be far less exciting than many of us imagine, but it's true. We can live with it. Wall Street can live with it, too, as Intel tries to take Altera public around 2026 to raise money to support its foundry business.

Reference link: 

https://www.nextplatform.com/2024/09/26/altera-is-being-realistic-about-fpga-compute-in-the-datacenter/

View more at EASELINK