The rise of the generative AI wave is driving the computing system to gradually transform from the traditional CPU-dominated mode to the GPU-dominated era.

While the AI boom has created a boom in GPUs, it has also made HBM, which plays a key role, very popular, becoming an emerging explosion outlet in the current AI track and a strategic high ground that technology giants are competing for.

Samsung, SK Hynix, Micron and other leaders in the storage industry have incorporated HBM into their core product lines, viewing it as the key to promoting technological innovation and market competition.

The popularity of HBM is causing huge waves in the storage market and even changing the current chip landscape. According to a Korean brokerage, SK Hynix's annual operating profit in the chip business this year may exceed Samsung Electronics for the first time.

01 Memory chips, prominent position

HBM's industry status soars

As we all know, AI system performance is highly dependent on memory capacity, bandwidth, and latency. Therefore, memory technology is becoming a more important design criterion when selecting infrastructure components such as GPUs and other accelerators.

It is understood that the center of gravity of data center revenue has shifted to GPUs, and these GPUs need to have faster memory throughout the memory hierarchy. Therefore, memory and storage and their suppliers are providing more value and occupying a more important position in the data center value chain.

Take HBM integrated into GPUs and accelerators as an example. HBM has now become a key factor in improving the performance of AI and high-performance computing systems.

For example, Nvidia's transition from H100 to H200 GPUs reflects this impact. The H200 equipped with 6xHBM3E (six-layer stacked HBM) provides twice the performance of H100 using 5xHBM3 for large language models (LLM). Specifically, the H200 has 141GB of memory and 4.8TB/s bandwidth, which greatly exceeds the 80GB and 3.35TB/s of the H100.

In addition, HBM is also the highest-proportioned part of AI chips. According to foreign media disassembly, Nvidia H100 costs nearly $3,000, of which SK Hynix's HBM accounts for the largest proportion, with a cost of about $2,000 directly exceeding manufacturing and packaging.

It can be seen that the significant increase in memory capacity and bandwidth shows that HBM is no longer an ordinary commodity, but a key driver of the performance of cutting-edge computing applications. Memory suppliers can now significantly distinguish the pros and cons of the final solution, unlike in the past when memory chips were just standardized commodities purchased mainly based on price.

Therefore, memory companies have transformed from commodity suppliers to key partners in promoting AI progress, with increasing influence on product development, pricing, competitive positioning and supply chain dynamics.

With the continuous increase in HBM capacity and the launch of new 12-layer high-stack products, companies such as Nvidia and AMD will position their products to a certain extent based on HBM functions. Due to the growing demand for HBM and its key role in AI acceleration, major memory manufacturers are prioritizing HBM production capacity and investment.

It can be said that memory suppliers have become important partners in the AI hardware supply chain. Currently, only SK Hynix, Samsung and Micron produce HBM.

HBM: Lack of Information

At the same time, driven by fields such as generative AI big models, HBM has gradually developed into one of the emerging hot industries. As upstream customers continue to issue new demands, the current HBM market is sending out signals of supply and demand.

According to TrendForce's estimates, the annual growth rate of HBM demand bits will be close to 200% in 2024, and will double again in 2025.

After more than a year of grabbing HBM production capacity, the word "lack" is still the top priority.

The HBM production capacity of storage manufacturers in 2024 and 2025 has been fully booked, and the order visibility even reaches the first quarter of 2026.

Industry analysis points out that there is a supply and demand gap in the HBM market in the short term, and it is expected that the HBM gap will reach 5.5% and 3.5% of production capacity this year and next year respectively.

In view of the current status and trends of the industry, the problem of tight HBM production capacity has become increasingly prominent. The production capacity of many high-performance GPUs cannot be increased, largely because HBM is not enough.

Faced with this challenge, manufacturers have accelerated the pace of capacity expansion.

SK Hynix has maintained its leading position in the HBM competition, and is also planning a substantial expansion of production capacity. It is understood that SK Hynix is making every effort to increase the production capacity of the fifth-generation 1b DRAM, and plans to increase the monthly production capacity from 10,000 pieces at the beginning of the year to 90,000 pieces at the end of the year, and further increase it to 140,000 to 150,000 pieces in the first half of next year, achieving a leap in production capacity.

Samsung is not to be outdone. After its HBM3 entered Nvidia's supply chain, the supply of HBM3e has been quite bumpy. Due to heat dissipation, power consumption and other aspects, the product has not yet knocked on Nvidia's door. But the Korean company said earlier that it expects HBM production capacity to expand nearly three times this year.

Micron has a dual-line layout in the United States and overseas, building advanced HBM test lines, and considering adding a production base in Malaysia to capture the huge market demand brought by the AI era. Micron's goal is to more than double its HBM market share to around 20% in 2025. Previously, the company also planned to build a new DRAM plant in Hiroshima, Japan, which is expected to start construction in early 2026 and be completed by the end of 2027 at the earliest.

It can be seen that the three major memory chip giants have shown their firm determination to expand HBM production capacity. It is particularly noteworthy that SK Hynix announced that it expects to invest US$74.8 billion by 2028, of which 80% of the funds will be poured into the research and development and manufacturing of HBM technology, and accelerate the mass production of the next-generation HBM4 chip by 2025.

Market research organization TrendForce pointed out at a recent industry seminar that as the world's top three HBM manufacturers continue to expand production capacity, global HBM production capacity is expected to increase by 117% year-on-year in 2025.

Behind the supply and demand relationship of supply and demand, HBM suppliers are making a lot of money. Even in the past few years when the storage market continued to be sluggish for several quarters, the HBM business still performed well and became the key to driving the demand of storage manufacturers.

This also means that the growing demand for HBM in AI applications has brought a more stable and growing market for memory suppliers, which may reduce the historical cyclicality of the memory industry.

Currently, TrendForce maintains its outlook for the DRAM industry, predicting that HBM will contribute 10% of the total DRAM bit output in 2025, doubling from 2024. Due to the high average unit price of HBM, it is estimated that the contribution to the total output value of the DRAM industry will exceed 30%.

In addition, Samsung plans to allocate about 30% of its DRAM production capacity to produce HBM, which may lead to a 13% reduction in global DRAM supply, thereby pushing up market prices.

As for the NAND Flash market, TrendForce said that after NAND Flash suppliers experienced huge losses in 2023, capital expenditures have become more conservative. The increasing demand for high-speed and large-capacity storage in AI applications will also drive the booming Enterprise SSD (eSSD) market. However, the demand for storage products such as DRAM and HBM will benefit from the AI wave and will squeeze out NAND Flash equipment investment in 2025.

According to the latest survey by TrendForce, NAND Flash products are affected by the weak peak season in the second half of 2024. The wafer contract price first fell in the third quarter, and the decline is expected to expand to more than 10% in the fourth quarter. In terms of module products, except for Enterprise SSD, which is supported by order momentum, it is expected to increase slightly by 0% to 5% in the fourth quarter; PC SSD and UFS buyers have more conservative procurement strategies because the end product sales are not as expected. TrendForce estimates that the overall contract price of NAND Flash products in the fourth quarter will decrease by 3% to 8% quarter-on-quarter.

TrendForce also pointed out that due to excessive inventory at module manufacturers and price-cutting competition from some original equipment manufacturers, NAND Flash wafer contract prices will experience a significant decline in the fourth quarter, with an estimated quarterly decrease of 10% to 15%, and the trend of expansion cannot be ruled out.

02 Memory chips, changing landscape

HBM changes the memory chip market

Behind the HBM capacity competition, there is a more intense technology competition that affects the direction of the chip landscape.

Samsung has long been the dominant player in the memory chip field. However, now under the HBM boom, Samsung is being attacked from both sides and is changing from a previous memory chip leader to a follower.

Samsung Electronics is on the verge of losing its title as the leader in memory chips, struggling to catch up with its rival SK Hynix while also fending off fast-catching Chinese companies.

Although Samsung Electronics reported a 274% increase in operating profit in the third quarter of 2024, the company is currently in a state of crisis. The looming crisis led Jun Young-hyun, the head of Samsung chips, to issue an unprecedented apology, admitting that it had not met market expectations, and to overhaul the organization.

Behind these financials lies a semiconductor strategy that is disintegrating, and reported profits that mask deeper problems, including missed market opportunities, competitive setbacks, and leadership changes that hint at internal turmoil, posing severe challenges to the South Korean technology giant.

Samsung's compliance committee also admitted in its annual report that Samsung is currently in a difficult situation and must deal with unpredictable and rapidly changing economic conditions around the world, loss of employee confidence, difficulty in recruiting talent, and technical challenges. It is fighting a tough battle.

At the heart of Samsung's predicament lies a series of missteps in its semiconductor business. Samsung's poor performance in the field of AI chips has caused its stock price to fall by more than 20% this year. Samsung has promised investors to invest heavily in technologies such as 12-layer HBM3E chips and hopes that its HBM3E chips will be approved by AI chip giant Nvidia, but there is no sign that Samsung's latest HBM products will pass Nvidia's qualification tests soon.

It is reported that the verification of Samsung's 12-layer HBM3E products has been hampered by heat dissipation problems and excessive power consumption. As a leading company in the field of AI hardware, Nvidia is known for its strict requirements for suppliers. After failing to deliver on time and problems were not resolved for several months, Nvidia appears to have purchased a large number of HBM3 and HBM3E products from SK Hynix.

Nvidia's data center market share soars, and so does HBM's value

It is foreseeable that losing key customers such as Nvidia could be a huge setback in the fast-growing memory market driven by AI.

While Samsung is solving its technical problems, SK Hynix and Micron have seized the opportunity to establish themselves as the preferred 12-layer HBM3E memory suppliers in the technology industry. This situation has damaged Samsung's reputation as a leader in memory chips and may have a long-term impact on it.

It is reported that SK Hynix is currently the only supplier of Nvidia's fourth- and fifth-generation HBM3 and HBM3E, while Samsung is still undergoing qualification tests. It can be seen that Samsung's efforts have not met expectations and have given competitors an advantage.

TrendForce said that from the perspective of HBM product types, since Nvidia's new products will gradually shift to 12-layer HBM3e starting with Blackwell GPU, HBM3e will replace HBM3 as a mainstream product in 2025, accounting for 89% of the overall HBM market demand, of which 12-layer stacked products will account for more than half, becoming the mainstream product that major AI manufacturers will compete for in the second half of next year, followed by 8-layer HBM.

This means that Samsung may continue to be widened.

Not long ago, Nvidia founder Huang Renxun announced that Nvidia's AI chip update cycle would be compressed from two years to one year. In other words, this strategy not only means that GPUs need to adapt to such rapid iterations, but also urges suppliers including memory chips to speed up the evolution of technology.

SK Hynix has recently begun to apply hybrid bonding to the mass production of 3D DRAM. TSV+ stacked bonding process is the current ideal solution for HBM; but as the number of stacked layers increases, heat dissipation requirements increase, and hybrid bonding is expected to become the next-generation HBM4 solution. In addition to SK Hynix, Samsung Electronics' advanced packaging team has also completed the verification of 16-layer hybrid bonding HBM memory technology, and 16-layer stacked hybrid bonds and technology will be used for HBM4 memory mass production in the future.

However, manufacturing HBM chips requires a lot of advanced technology, but the prerequisite is the quality and cost of DRAM, which depends on its yield. The yield of DRAM is crucial for HBM, because if it does not meet the standards, the cost and processing time of HBM will inevitably deteriorate.

After the DRAM is qualified, the next step is which packaging and bonding technology the company will use to determine the performance of HBM. Samsung Electronics and SK Hynix are both using 1b nanometer process technology (equivalent to the 12 nanometer range) for HBM3E DRAM. Yield is usually confidential data for each memory manufacturer, but it is revealed that Samsung Electronics' 1b nanometer DRAM yield is significantly lower than SK Hynix.

On the other hand, as the competition between China and the United States escalates, Chinese memory chip manufacturers have been striving to achieve chip technology independence for many years, and the speed at which Chinese memory chip manufacturers expand their market share is also increasing, and Samsung Electronics is also facing new challenges.

According to data released by TrendForce, the market share of Chinese chip manufacturers in the DRAM market is expected to reach 10.1% in the third quarter of next year, up from 6% this year. A recent report by Morgan Stanley predicts that the DRAM production of domestic DRAM memory manufacturers will exceed 10% this year, and predicts that the invasion of Chinese manufacturers into the traditional DRAM market will be faster.

Samsung, "Winter is coming"?

Noh Geun-chang, director of Hyundai Securities Research Center, said that the overall semiconductor market will not face a cold winter due to strong demand for AI-related semiconductors, but the cold winter has already arrived for Samsung, as it has failed to prove its competitiveness in the HBM field.

Samsung Electronics investors are also going through a difficult time. It is reported that Samsung Electronics' stock price rose to 87,000 won in early July, but fell to 59,300 won on October 11. It is not only the falling stock price that makes investors uneasy, because stock prices fluctuate according to market conditions, and the stock price trend shows that Samsung Electronics' competitiveness has turned red.

Not only in the HBM field, Samsung's other two major businesses, smartphones and foundry, are also facing severe competition.

In terms of US dollars, the chip giant's sales growth rate for 10 years has averaged only 1% per year. Apart from the acquisition of automotive equipment giant Harman in 2017, Samsung's business has not changed much. For a company that has been selling the same products for 10 years and sales have stagnated, there is little room for the stock market to give high praise.

To meet these challenges, Samsung recently carried out a major corporate restructuring. Executives from the memory, foundry, system LSI, and manufacturing divisions were replaced, and new leadership was appointed to key positions. Although Samsung made this move as a strategic move to show that it takes these issues seriously. However, simply replacing management may not be enough to solve the fundamental problems in product delivery and quality.

In May this year, in the foundry field dominated by TSMC, Samsung tried to attract customers such as AMD and Apple by promising better performance than TSMC. However, due to Samsung's low production yield, the reality is not optimistic. The continued yield problems have disrupted the production foundry business, causing potential customers such as AMD and Apple to rely on TSMC instead.

Samsung's current goal is to improve yield, regain customer trust, and prove that Samsung's foundry ambitions are credible. But given TSMC's good reputation and stable performance in advanced node manufacturing, this will be a challenge for Samsung.

Samsung also faces the daunting task of correcting the HBM3E problem and ensuring that Samsung can provide products that meet Nvidia's strict standards in DRAM technology. This goal has reportedly been difficult to achieve so far, with top priorities including addressing heat and power issues and preventing Samsung HBM from encountering similar setbacks in future iterations. Staying ahead of market trends is critical to regaining a competitive edge.

Collectively, these issues are significant obstacles for Samsung's semiconductor division.

Whether it's Nvidia's decision to work with SK Hynix or AMD, Apple and other companies' chips switching to TSMC, this not only means that Samsung has lost an opportunity to cooperate; it also shows the industry that Samsung may no longer be the reliable memory supplier and wafer foundry it once was.

SK Hynix and Micron are meeting the strong demand of the AI market with HBM chips, while Samsung is still struggling with technical issues and delays; TSMC is outperforming Samsung in the foundry business, continuing to deliver 3nm chips to customers, while Samsung is still struggling with yield issues.

Therefore, in order to change the status quo, Samsung must not only solve the heat and power issues affecting its HBM memory, improve 3nm production yields, and more importantly, rebuild customer trust: to regain the trust of major industry players, consistent and reliable performance is required. Samsung may need to adopt a strategy of under-promising and over-delivering, focusing on delivering on promises rather than just making bold claims.

But will Samsung eventually overcome these challenges? It is too early to predict.

But its path forward will certainly be difficult, as competitors have already established strong positions in key markets. However, the semiconductor industry is changing rapidly, and with the right strategy and perfect execution, Samsung can strive to regain its leadership; if it fails to effectively address its current shortcomings, Samsung may be relegated to a secondary position in the market where it once dominated.

Last words

As the Science and Technology Innovation Board Daily said: Looking back 11 years ago, SK Hynix had already manufactured HBM for the first time, and welcomed its first customer AMD two years later. But after that, HBM was rarely bought by customers due to its high price, and it was silent for many years, so it was once regarded as "the wrong technology tree".

Today, Nvidia's GPU has finally "rectified" HBM, and the entire industry is catching up with the progress of HBM. After all, in the AI hardware arms race, time is life, and falling behind means elimination.

On the journey of HBM technology, a war without gunpowder is unfolding, and it also witnesses the reshaping of the competitive landscape of the chip market.

View more at EASELINK