TSMC warns AI chip crunch will last another 18 months

Bad news for anyone looking to get their hands on Nvidia's top specced GPUs, such as the A100 or H100: it's not going to get any easier to source the parts until at least the end of 2024, TSMC has warned.

The problem, it seems, isn't that TSMC – which fabricates not just those GPUs for Nvidia but also components for AMD, Apple, and many others – can't make enough chips. Rather, a lack of advanced packaging capacity used to stitch the silicon together is holding up production, TSMC chairman Mark Liu told Nikkei Asia.

According to Liu, TSMC is only able to meet about 80 percent of demand for its chip on wafer on substrate (CoWoS) packaging technology. This is used in some of the most advanced chips on the market today – particularly those that rely on high-bandwidth memory (HBM) which is ideal for AI workloads.

Liu expects this is a temporary bottleneck in the production of machine-learning accelerators and that additional CoWoS capacity should come online within a year and a half. Incidentally, TSMC recently announced plans to expand its advanced packaging capacity in Taiwan with a $3 billion facility at the Tongluo Science Park in Miaoli County.

Until TSMC can bring additional capacity online, Nvidia's H100 and older A100 – which power many popular generative AI models, such as GPT-4 – are at the heart of this shortage. However, it's not just Nvidia. AMD's upcoming Instinct MI300-series accelerators – which it showed off during its Datacenter and AI event in June – make extensive use of CoWoS packaging technology.

AMD's MI300A APU is currently sampling with customers and is slated to power Lawrence Livermore National Laboratory's El Capitan system, while the MI300X GPU is due to start making its way into customers' hands in Q3.

We've reached out to AMD for comment on whether the shortage of CoWoS packaging capacity could impact availability of the chip and we'll let you know if we hear anything back.

• After failed takeover, Intel and Tower Semi aren't giving up on the relationship
• Rapidus ramps as construction begins on 2nm wafer fab
• China cooks covert chips, recruits global geeks to dodge US restrictions
• Our AI habit is already changing the way we build datacenters

It's worth noting that TSMC's CoWoS isn't the only packaging tech out there. Samsung, which is rumored to pick up some of the slack for the production of Nvidia GPUs, has I-Cube and H-Cube for 2.5D packaging and X-Cube for 3D packaging.

Intel, meanwhile, packages several of the chiplets used in its Ponte Vecchio GPU Max cards, but doesn't rely on CoWoS tech to stitch them together. Chipzilla has developed its own advanced packaging tech, which can work with chips from different fabs or process nodes. It's called embedded multi-die interconnect bridge (EMIB) for 2.5D packaging and Foveros for vertically stacking chiplets on top of one another. ®