Samsung Electronics (005930.KS) has ended up working with TSMC, its rival in contract chip manufacturing, on the development of custom high-bandwidth memory for Nvidia because artificial intelligence chips are running into a memory bottleneck in both performance and supply. With HBM data transfer speeds unable to keep pace with the computing speed of graphics processing units, the plan is to lift HBM performance using TSMC's advanced logic process and to add TSMC's production capacity to increase supply.
Nvidia is working with Samsung Electronics and TSMC to custom-design the base die, a core component of its own HBM product called NVHBM, according to industry sources on the 18th. The base die sits at the bottom layer of an HBM stack and is made up of logic circuits that determine data transfer speeds. How it is designed shapes the competitiveness of the final product.
The custom base die for NVHBM is notable for incorporating the memory controller that previously sat on the GPU. By shifting that supporting function to the NVHBM base die, Nvidia freed up space on the GPU to add more computing circuits and boost performance. To make room for the controller on the base die, however, the area taken up by the physical layer, or PHY, the circuitry that exchanges electrical signals with external chips, was cut by 60% from standard HBM. As a result, NVHBM can widen memory bandwidth by up to 30% and cut power consumption by 15% compared with standard seventh-generation HBM4E.
Nvidia's demand for TSMC's 3-nanometer logic process for the NVHBM base die is tied to that design approach. Packing functions moved over from the GPU into the base die requires an ultrafine 3-nanometer-class process capable of integrating more circuits into a limited area. Holding down production costs also requires a high yield, or the share of working chips on a single wafer.
The industry puts the yield of TSMC's 3-nanometer process at more than 90%, compared with around 60% for the comparable 2-nanometer process at Samsung Foundry, known as SF2. For Nvidia, consolidating base die production at TSMC is the more stable choice.
A bigger factor than technical conditions is that HBM supply is not keeping up with demand. HBM demand next year is forecast to reach about 61.5 billion gigabits, a measure of storage capacity, up 91% from this year, according to investment bank UBS. New shipments from memory makers, by contrast, are expected to total only 54.6 billion gigabits, leaving a shortfall of roughly 10% against demand.
Samsung Electronics needs to raise output of both core components of HBM, the core die and the base die, to match rising demand. The core die, which functions as the memory chip itself, is produced using the advanced sixth-generation 1c DRAM process at Line 17 of the Hwaseong campus and at P2, P3 and P4 at the Pyeongtaek campus. P5-1, which is under construction with a target of starting up the year after next, will also run mainly on 1c lines.
Base die capacity, however, is relatively limited. Samsung Electronics' 4-nanometer base die production lines are located only at Hwaseong Line 17 and parts of P3 at Pyeongtaek. Under the current production setup, which was built around existing HBM demand, it is difficult to absorb additional volumes of custom HBM such as NVHBM.
TSMC has the capacity to cover the base die shortfall that Samsung Electronics cannot handle. In its third-quarter earnings announcement, TSMC said it would add two new 3-nanometer production lines at its Fab 18 in Tainan, Taiwan, with mass production starting in the first half of next year and the first half of the year after next, respectively. It also plans to install 3-nanometer lines at its second plant in Arizona and its second plant in Kumamoto, Japan, targeting mass production in the third quarter of next year and in 2028.
That does not mean base die production has been locked in with TSMC. While using TSMC's base die for NVHBM at Nvidia's request, Samsung Electronics is also preparing to produce the component at its own foundry in anticipation of continued demand growth.
"TSMC's 3-nanometer lines also make high-performance computing and smartphone chips, so allocated volumes could reach their limit," an industry official said. "Samsung Foundry is therefore validating NVHBM base die production on its 2-nanometer process, which is comparable to TSMC's 3-nanometer process."
Another industry expert said the joint development of Nvidia's NVHBM has broadened cooperation between Samsung Electronics and TSMC into a relationship that produces core HBM components through advanced packaging processes. "The two companies, which worked together on TSMC's CoWoS packaging process for AI chips four years ago, joining hands amid a global memory shortage carries significant implications for the industry," the expert said.