At the 2024 Developer Conference some time ago, the Peripheral Component Interconnect Special Interest Group (PCI-SIG) announced the latest progress of PCIe 6.0 and PCIe 7.0.

PCIe 6.0 deep compliance testing will start in 2025, and the final specification of PCIe 7.0 is still scheduled for release in 2025, but the corresponding Live Compliance program has been postponed to 2028. Judging from the reactions of all parties at the recent PCI-SIG Developer Conference, everyone's patience seems to be gradually wearing out.

These products will initially be used in data centers, then industrial, automotive, aerospace and military applications, and then consumer electronics, so delays in any link may cause delays in the entire roadmap.

PCIe, as a high-speed serial computer expansion bus standard, has now been popularized in every PC. Whether it is ordinary users or professionals, whether it is large companies or small businesses, they are all enjoying the benefits of this standard.

But this long-standing standard is not immutable. In the past two decades, it has undergone six iterations, and the transmission rate per channel has increased from 2.5 GT/s to 128 GT/s, which has achieved earth-shaking changes. The latest PCIe 7.0 solution has a x16 bidirectional bandwidth of up to 512GB/s, which is further doubled.

01 What is PCIe?

PCIe is the abbreviation of PCI-Express, which is a high-speed serial computer expansion bus standard.

The English word for bus is Bus. Bus is understood as car in China, and there is another explanation in China that it means bus. In fact, these two translations still have something in common. For example, Bus is an important way of communication between two cities, while Bus is an important way of communication between computers.

If we want to go from Tianjin to Beijing to do something, we can take a long-distance bus, and the computer needs to go through the PCIe Bus. Simply put, the Bus is a communication highway between multiple hardware.

Corresponding to the highway, PCIe also has different specifications. Just like the highway, different roads have different speed limits. For example, the width and maximum driving speed of national highways and highways are different. This corresponds to the specifications of PCIe1.0, PCIe2.0, PCIe3.0, and PCIe4.0.

But even for the same business, there are different speed limits on different routes. This is similar to the performance of different expansion slots of PCIe1.0. For example, the maximum transmission speeds supported by the four expansion modes of PCIE x16/x8/x4/x1 are also different.

It can be said that the PCIe bus is an important part of the computer device tree. Almost all peripheral hardware expansions require PCIe. Today we will talk about the development history of PCIe.

02 Digging into the ancestors of PCIe

Many years ago, the bus interfaces used by different devices on the computer motherboard were completely different. The hard disk had a corresponding bus interface, and the network card had a corresponding bus interface.

The advantage of this is that each device has a corresponding bus interface, and it is simple to optimize performance. But for individuals, with the gradual increase in the number of expansion devices, if the corresponding bus interface is expanded on the motherboard, it is a bit of a loss, and it also causes the limitation of motherboard expansion and brings a lot of inconvenience to the unification of hardware specifications.

In order to solve this bottleneck, IBM and Intel joined forces and adopted the first generation of ISA slots on the first generation of IBM PC XT models in 1981. As the pioneering work of modern PC, the 8-bit ISA provided a bandwidth (or transfer rate) of 4.77MB/s.

Due to its good compatibility, the ISA bus was welcomed by various manufacturers as soon as it was launched, and became the most widely used system bus in the 1980s. However, since ISA uses a parallel bus and the anti-interference technology at that time was not mature, the bandwidth of the ISA bus could not be designed very high, and could only reach a rate of 8MB/s.

This transmission speed was still a bit "not powerful enough" for the large data transmission needs such as images that gradually emerged in the 1990s.

In addition to the disadvantage of slow speed, the devices on the ISA bus interface cannot be automatically configured and cannot be plug-and-play. In addition, IBM introduced the MCA bus in the PS/2 product line, forcing several other PC compatible machine manufacturers to work together to come up with EISA. All these led to the eventual elimination of the ISA bus.

In 1992, Intel came up with its own killer move, proposing the PCI (Peripheral Component Interconnect) bus protocol, and relying on its strong influence in the PC field, it convened a group of friends to form a corporate alliance called PCI-SIG (PCI Special Interest Group). Since then, this organization has been responsible for the standard setting and promotion of PCI directors.

I have to praise Intel's foresight. It started to build its own "circle of friends" in the 1990s. Compared with the closed IBM, Intel's win-win mentality has made the PCI standard widely promoted and used. Unified standards are also conducive to the innovation of peripheral device manufacturers. Since then, various PCI devices have emerged and enriched the entire ecological environment of PC.

Compared with the ISA bus, the bandwidth of the PCI bus has been upgraded to 132MB/s, and the speed improvement is very obvious. In addition, it supports automatic configuration and plug-and-play.

But the PCI bus is not perfect. Both the PCI bus and the ISA bus use parallel bus design, so the transmission speed is also affected. Then there is the bandwidth sharing mechanism of the PCI bus. Under high load, other devices may compete for bandwidth, and hot plugging is not supported.

So in order to solve the defects of the PCI bus, the technology was upgraded and innovated again. In 2004, Intel once again led its partners to revolutionize PCI, which is the PCI Express (PCIe for short) bus mentioned in this article. Today, computers have begun to be the fifth generation (gen5, 5.0), and they are even more indispensable in computers.

Nowadays, PCIe can support a lot of devices, including graphics cards, solid-state drives (PCIe interface form), wireless network cards, wired network cards, sound cards, video capture cards, PCIe to M.2 interface, PCIe to USB interface, PCIe to Type-C interface, etc.

03 PCIe transmission speed and width

There are two forms of PCIe bus, one is the interface and the other is the channel. When PCIe exists in the form of an interface, it is a long horizontal slot on the motherboard. Some people may ask, what is the function of this long slot, why are the long slots on my motherboard of different lengths?

These long slots are actually different interfaces of PCIe. Common PCIe interfaces mainly have four sizes, X1, X4, X8, and X16. Generally, the maximum bandwidth of the four sizes of slots is different. Their speeds are different. The PCIe speed of X16 is twice that of X8, and X8 is twice that of X4. Of course, the above paragraphs are all based on the same generation of PCIe bus.

So how is the throughput of PCIe calculated? First, let's look at the calculation formula for different PCIe interface sizes: Throughput = Transmission Rate * Coding Scheme * Physical Channel Lane

Take PCIe4.0x4 as an example. This series is the 4.0 version of PCIe, which contains 4 physical channel Lanes. The throughput of each channel is:

16GT/s x 128b/130b = 1.969GB/s

So the throughput of PCIe4.0x4 is: 1.969GB/s x 4 = 7.877GB/s. If it is PCIe4.0x16, the maximum throughput is 64GB/s.

In addition to different throughputs, what are the differences in the applications of different PCIe sizes? Let's take a closer look at the applications of X1, X4, X8, and X16.

PCIe x16 slot: As shown in the figure above, the PCIe x16 slot is 89mm long and has 164 pins. There is a slot on the outer side of the motherboard, which divides 16x into two groups, front and back. The shorter slot has 22 pins, which are used for power supply. Each size of slot has this 22-pin power supply design. The longer slot has 142 pins, which is mainly used for data transmission and has the high bandwidth brought by 16 channels.

At present, the PCIe x16 slot is mainly used for GPU graphics cards and RAID array cards. This slot has excellent compatibility and can be backward compatible with x1/x4/x8 level devices, and has stronger transmission performance. It can be said that the PCIE x16 slot is a universal slot for PCIE.

Since PCIe x16 slots are commonly used for graphics cards, they are directly connected to the CPU processor and are physically close to the CPU, so the data exchange between the graphics card and the processor can reduce latency and fully utilize the system's performance.

PCIe x8 slot: 56mm in length, with 98 pins. Compared with PCIe x16, the data pins are reduced to 76, and the short power pins are still 22 pins.

For compatibility, PCIe x8 slots are usually processed into the form of PCIE x16 slots, but only half of the data pins are valid, which means that the actual bandwidth is only half of the real PCIE x16 slot. You can observe the motherboard wiring, and there is no line connection in the second half of the x8, and even the pins are not soldered.

In fact, except for flagship or server motherboards, which can provide multiple real PCIex16 slots, most motherboards only provide one real PCIe x16 slot, which is the one closest to the CPU. The second and third PCIe x16 slots are mostly PCIe x8 or even x4 level.

PCIe x4 slot: 39mm in length, also based on the PCIE x16 slot, to reduce the data pins, mainly used for PCIe SSD solid state drives, or M.2 SSD solid state drives installed through PCIE adapter cards.

PCIe x4 slots are usually extended from motherboard chips, but with the increase in the number of PCIE channels inside the CPU, some high-end motherboards can now provide PCIE x4 slots directly connected to the CPU for installing PCIE SSD solid state drives.

However, like the PCIe x8 slot, for compatibility, most PCIe x4 slots are now made into PCIe x16 slots, or expanded into M.2 interfaces for installing M.2 SSDs, M.2 wireless network cards or other M.2 interface devices, and the rest of the expansion cards are left to the PCIE x1 slot.

PCIE x1 slot: The length is only 25mm, and its data pins are greatly reduced to 14 compared to the PCIE x16 slot. The bandwidth of the PCIE x1 slot is usually provided by the motherboard chip. The main purpose is that independent network cards, independent sound cards, USB 3.0/3.1 expansion cards, etc. will use the PCIE x1 slot. You can even install a graphics card to the PCIE x1 slot through an adapter cable for mining or multi-screen output.

Another important function of X1 is to replace the original PCI devices.

04 What is the future of PCIe?

Since the development of PCIe interface in 2001, it has established a high enough "moat" in the integrity of the protocol. In the short term, no enterprise will have the motivation to redefine an interface protocol to surpass PCIe in performance. On the other hand, there is no foreseeable prototype innovation in the dimension of technology.

Generally speaking, it takes 1 to 2 years from the specification to commercialization, just like Moore's Law estimates that every 1 to 2 years, the product needs to be upgraded and evolved.

However, Intel relies on its monopoly advantage and follows suit in PCIe upgrades. After the release of PCIe1.0 in 2004, PCIe5.0 began to gradually appear on motherboards until the release of the new generation of servers in 2023.

And it can be said that PCIe 5.0 is an important watershed in the development of PCIe. This is because man's calculations are not as good as God's calculations. Intel did not expect the sudden outbreak of demand for big data AI, visual rendering, genetic analysis, and EDR simulation, which made PCIe data transmission unable to keep up with the demand of GPU.

In order to solve the limitation of PCIe in data transmission, several different transmission and memory semantic protocols gradually appeared in the market a few years ago - IBM's OpenCAPI memory interface protocol, Xilinx's CCIX protocol, NVIDIA's NVLink protocol, HP Enterprise Edition's Gen-Z protocol, all of which are to solve the problem of slow transmission of PCIe4.0.

Seeing that everyone is revolutionizing PCIe, Intel launched the CXL (Compute Express Link) protocol interface in March 2019, encapsulating the CXL protocol into PCIE link layer data packets for transmission, and diverting CXL exclusive transactions to CXL processing logic according to transaction identifiers at the PCIE master control backend on the CPU side.

Intel hopes to achieve high-speed and efficient interconnection between CPU and GPU, FPGA or other accelerators to meet the requirements of high-performance heterogeneous computing. The most noteworthy thing is that the interface specifications of the CXL standard are compatible with PCIe 5.0, so that the CXL protocol can run on PCIe5.0, further consolidating the influence of PCIe in computers.

Intel's strategy is that since the demand trend of CPU and other demand has already arrived, it is better to use GPU, DPU, etc. as a knife to form a certain balance with NVIDIA. Now, NVIDIA has also joined the CXL Alliance. For Intel, I have built a "highway" to memory anyway, and your GPU has to listen to me.

In this way, the development of GPU can be further checked. How many GPUs a CPU wants to support must be decided by Intel.

It is worth noting that each processor of the fourth-generation Intel Xeon Scalable Processor supports up to 4 CXL devices, supporting CXL Type 1 and CXL Type 2. ) These will enhance the comprehensive capabilities of servers and provide higher value for memory-intensive and IO-intensive scenarios.

05 Summarize

PCIe 7.0 is the next generation of computer interconnect technology, designed to increase data transfer speeds to 128 GT/s per pin, double the 64 GT/s of PCIe 6.0 and quadruple the 32 GT/s of PCIe 5.0. This will allow a 16-lane (x16) connection to support 256 GB/s of bandwidth in each direction simultaneously (excluding encoding overhead). Such speeds will be very convenient for future data centers as well as artificial intelligence and high-performance computing applications that require faster data transfer rates, including network data transfer rates.

To achieve the impressive data transfer rates, PCIe 7.0 doubles the bus frequency of the physical layer compared to PCIe 5.0 and 6.0. In addition to this, the standard retains the pulse amplitude modulation with four-level signaling (PAM4), 1b/1b FLIT mode encoding, and forward error correction (FEC) technology that has been used in PCIe 6.0. In addition, PCI-SIG said that the PCIe 7.0 specification also focuses on enhanced channel parameters and coverage, as well as improved power efficiency.

It is worth mentioning that last summer, PCI-SIG announced that it would explore the possibility of PCIe fiber optic connections. To this end, the PCI-SIG Optical Working Group was established in August 2023 to design appropriate form factors for connectors and transceivers. Optical connection technology that transmits data via light waves has the potential to expand the application areas of PCI Express, such as cloud computing, high-performance computing, and quantum computing. Compared with transmission over copper wires, this is expected to achieve higher throughput, lower latency, and lower energy requirements.

For these scenarios, the PCIe 7.0 interface will likely reach a data transfer rate of 512GT/s, which is of great significance to heterogeneous computing architectures.

View more at EASELINK