PCIe technology will become an important part of the AI infrastructure market.

According to ABI Research's "PCI Express Market Vertical Opportunities" report, the total addressable market (TAM) for PCIe technology in the AI space is expected to grow from $449.33 million to $2.784 billion by 2030, with a compound annual growth rate (CAGR) of 22%. One emerging use case for AI is generative artificial intelligence, or GenAI. GenAI is an AI technology used to generate content, including text, images, video, audio, and more.

As GenAI has evolved, some of the unique challenges in GenAI applications have become clear, such as the need for low-power, low-latency, robust technology to connect these systems together. Due to the increasing complexity and size of large language models (LLMs), state-of-the-art generative AI models cannot fit on one GPU, one server, one rack, or even one data center.

PCIe technology offers numerous advantages for generative AI applications, as its inherent DNA is well suited to implementing factorized systems, including the distributed multiplication capabilities that are valuable for LLMs. In this article, we will discuss how PCIe technology is currently being used in generative AI, how PCIe technology capabilities are perfectly suited to meet the growing demands of AI, and how the relationship between PCIe technology and AI will continue to evolve in future applications.

PCIe technology features meet the technical requirements of generative AI

PCIe technology is a ubiquitous I/O interconnect that provides the fabric that connects nodes together by enabling low-latency, low-power connectivity while always ensuring backward compatibility. PCIe technology connects entire data centers, creating pooled resources of compute, memory, and storage to meet the unique and specific needs of generative AI applications.

As demand for higher data rates grows and the industry moves from NRZ to PAM4 signaling, forward error correction (FEC) is critical to maintaining reliability. PCIe addresses this issue for generative AI and all low-latency applications by utilizing FLIT (Flow Control Element) mode. FLIT helps maintain the low latency of PCIe technology while still providing low post-FEC error rates. In addition, the PCIe architecture includes low-power modes that save energy when less data throughput is needed and more energy when the link is temporarily unused through L0p and L1 substates.

Further exploring the importance of PCIe's low-latency advantage, hardware coherency plays a vital role in scaling networks to improve efficiency. It's not just about overall bandwidth – delays in data exchange can cause GPUs and CPUs to stall while they wait for data. Pipeline algorithms often rely on distributed results, and even a single node's latency can cause significant slowdowns, idling valuable compute resources. PCIe technology now features FLIT mode to minimize data transfer latency and keep it consistent for efficient performance.

Emerging PCIe Technologies Power the Future of Generative AI

PCIe technology will help advance the adoption of generative AI as it extends to both electrical and optical links in the backend networks where AI runs. As bandwidth increases and electrical reach decreases, CopprLink™ internal and external cables can extend the reach of PCIe signals in generative AI applications, with CopprLink internal cables providing up to 1 meter of reach within a single system and CopprLink external cables providing up to 2 meters of reach. Additionally, the PCI-SIG Optical Working Group is currently investigating ways to implement PCIe technology over optical links to ensure that any PCIe link is viable in future generative AI applications.

Source: Content from awavesemi

View more at EASELINK