As an equity analyst known to frequent the local pubs in the Square Mile, I might be forgiven for assuming that the whole world is feverishly monitoring developments in data centres and the semiconductor sector. Yet, as a quick scan of the non-financial broadsheet and tabloid media would illustrate, this obsession remains confined largely to the pink pages of the Financial Times.
Headlines focus on UK politics and celebrity gossip. Yet the state of the data centres powering AI will affect the next ten years more than most things Andy Burnham will do.
The fact data centres and semiconductors don’t often make it to the front pages might have something to do with the overwhelming level of complexity. The supply chain which feeds a data centre is one of the most complex in the world. With this complexity comes supply chain bottlenecks. As AI demand has come to dominate the sector, the supply chain is now best thought of as a series of bottlenecks which are constantly shifting and emerging as it tries to keep pace with the insatiable demand currently being generated by AI.
Power
Data centres power both the training and inference (usage) of large language models (LLMs) such as ChatGPT or Claude – and AI workloads are fundamentally power hungry. The advent of LLMs has created a huge demand for data centre power that far outstrips any previous demand for more traditional data centre workloads such as database storage.
Demand has thus outstripped the ability of new power to come online. Focusing on the US specifically, where much of the world’s data centre workloads currently reside, the supply of grid power cannot keep up.
The supply chain is now best thought of as a series of bottlenecks which are constantly shifting.
Data centre size is now most commonly described in terms of gigawatts (GW), i.e. the amount of power available to a data centre per year. According to current industry estimates, a 1GW data centre costs c.$38bn in upfront capital expenditure. Of this, the technical equipment inside the data centre accounts for roughly two thirds of this spend, with much of this going towards semiconductors.
Given the pace at which data centres are being built, the grid cannot keep up. In the US, the lion’s share of grid capacity comes from gas. Delays stem from a range of problems, from securing permits to supply chain blockages, and these have been compounded by the fact that the preferred energy source, combined cycle gas turbines (CCGTs), have the slowest build times despite being the most efficient.
Estimates point to OpenAI and Anthropic being able to generate over $100bn in revenue per GW per year. Getting a data centre up and running even two months earlier than forecast can therefore translate into tens of billions of dollars. Hence, it’s no surprise that many data centre operators are increasingly using ‘behind-the-meter’ (BTM) power: on-site forms of generation that are co-located with the data centre. Elon Musk’s xAI pioneered this technique and built a data centre by bypassing the grid and generating on-site power using gas turbines and engines mounted on trucks. He has even resorted to building data centres on state borders so that if permits can’t be acquired quickly enough in one state, power infrastructure can be built in a neighbouring state where permitting is more prompt.
Fundamentally, with this amount of revenue on the line, solutions will be found and BTM will likely account for a large portion of incremental future supply. And yet, as Musk recently affirmed on a SpaceX earnings call, as companies find energy solutions they face another significant blockage: semiconductors.
No logic
The servers that make up the core of AI data centres need, amongst a litany of other parts, several types of semiconductors. Two of the most important are logic chips and memory chips. Logic chips are the brains of the servers, with the most common types used in AI being Graphics Processing Units (GPUs) and Central Processing Units (CPUs). These chips are designed by companies such as Nvidia and AMD, however they are generally manufactured by a Taiwanese company called Taiwan Semiconductor Manufacturing Company (TSMC).
TSMC has a near monopoly on leading-edge manufacturing capacity. The complexity of the chip fabrication process is described in ‘nodes’. Nodes historically referenced the width of the smallest feature on a microchip but today the term is used as more of a marketing term to describe generations of semiconductor manufacturing. Demand for TSMC’s N3 node is very strong, with Nvidia, AMD, Alphabet, Amazon, Meta and Microsoft all competing for capacity.
Given the pace at which data centres are being built, the grid cannot keep up.
For any company hoping to manufacture a leading-edge AI chip, TSMC is really the only option. Demand for manufacturing capacity has been outstripping supply for some time and only recently has TSMC started to really ramp up spending on more capacity. Building leading-edge chip manufacturing facilities takes time and so, silicon supply looks set to be tight for the foreseeable. The prospect of another company competing with TSMC at the leading edge in the near term seems remote, therefore the rate at which TSMC can build will remain a key constraint on data centre supply.
Memory problem
The other essential part of AI servers, memory chips, has seen prices soar as rapid demand caught industry players off guard. Memory comes in two broad forms: volatile and non-volatile. Non-volatile memory stores information even when turned off; volatile memory does not. Both types of memory are used in data centres, but volatile memory has seen a particularly acute crunch in recent months.
A type of volatile memory essential to running AI workloads is Dynamic Random Access Memory (DRAM). A subset of DRAM called High Bandwidth Memory (HBM) is particularly sought after for use in AI servers. DRAM capacity is expanding and DRAM can be manufactured by a wider group of companies; Micron, Samsung and SK Hynix are notable examples. And yet, demand continues to outstrip supply for a number of reasons. One crucial reason is that HBM requires around three times as much manufacturing capacity to create the same amount of storage as traditional DRAM. This comes at a time when AI servers are demanding even greater densities of HBM. This is therefore crowding out traditional DRAM, for which there is also strong data centre demand.
The net result has been a sharp acceleration in memory prices. So much so, that it is rumoured that chip designers such as Nvidia, which uses HBM alongside its GPUs to create AI hardware, are considering cutting back the amount of HBM allocated to their next generation system.
As companies find energy solutions they face another significant blockage: semiconductors.
As with logic manufacturing, the memory foundries (manufacturers) are looking to expand capacity but as yet don’t seem to be able to satiate demand. When supply might be able to meet demand is the subject of constant debate, but this is just one of a number of headaches facing anyone trying to build a data centre.
Beyond power, logic and memory, many other areas of tight supply exist; such is the reality when a market is growing so quickly. Navigating these shortages and finding solutions is becoming a key differentiator and could represent billions in revenue lost or gained.
The outlook for AI demand is fundamentally uncertain, and with this huge build out of data centre capacity comes the risk of overbuild. If demand does not continue to grow at a rapid pace, those fighting shortages to build data centres may find themselves not generating adequate returns on their investments. And yet, as it currently stands, demand continues to outstrip supply. Whilst this continues, there will be a premium price offered to anyone who offers a solution.
