The Compute Stack

The tectonic plates within the technology sector are rapidly shifting, catalyzed by the rapid rise of AI over the past six months. AI is causing a Cambrian explosion in infrastructure as the demand for “Compute” – which is the underlying fuel behind AI, Bitcoin mining, trading, graphics rendering, and more – is in a one-way trend higher. Demand for Compute is vastly exceeding supply, putting strain on the “Compute Supply Chain.” In BitOoda’s “High Performance Compute Primer” published on 5/12/23, we defined Compute and provided an overview of the broader Compute ecosystem. In this report, we will focus on the underlying infrastructure forming the supply side of Compute, which we refer to as the “Compute Stack.” ​

Data centers form the foundation of the Compute Stack. Data centers, which have been regarded as stable-growth infrastructure plays with parallels to the real estate industry, have suddenly become coveted growth engines. The “digital real estate” provided by data center infrastructure is now massively in-demand, positioning these data centers at the nexus of AI’s ascent.​

Hardware players also are thriving in the Compute ecosystem. The most prominent example is NVIDIA, which made its bet on the AI boom years (if not decades) ago and therefore is dominating the market for next-generation GPUs (A100s, H100s), which are being used to train Large Language Models (LLMs) in the still-nascent AI boom. NVIDIA therefore has (temporarily) uncontested pricing power, which, coupled with a surge in demand, is setting it up as a winner. However, as Bezos has said, “your margin is my opportunity,” and it is likely that other hardware players will develop hardware acceleration solutions to compete. ​

Cloud Service Providers, namely “hyperscalers,” comprise the front lines of the Compute Stack. The AI revolution and ensuing surge in Compute demand is occurring from the top-down, with the largest hyperscalers (AWS, Google Cloud, Microsoft Azure) capturing the initial value from AI model infrastructure (for example, Microsoft’s recent deal with OpenAI). However, like with hardware players, we expect increased competition for hyperscalers over the long run.​

Power producers are the engine room of the Compute space. The least talked-about (yet still significant) component of the Compute Stack is the underlying power producer. It is ironic that power consumption of Bitcoin mining was an immediate attack vector against BTC, while the surging demand for power to train and operate AI models has not come into the spotlight. Nevertheless, the AI industry could be magnitudes larger than BTC (since AI permeates all industries), and the power requirements will be significant. Ultimately, we expect that AI data centers and other players in the Compute Stack will adopt renewable energy mandates.​

The Compute Stack will rapidly evolve. A generic philosophy is that in technology, “change is the only constant,” and we believe this will apply to the Compute Stack. AI’s mainstream revolution is so new, and the emerging demand drivers for Compute (e.g., zero knowledge proofs) are on the horizon and rapidly approaching. As the Compute industry scales, we could see disruption across all layers of the Compute Stack, with decentralization of data centers, competitors in the hardware acceleration arms race, and disintermediation of the hyperscalers. ​

The Compute Stack - Data Centers

• The data center was historically viewed as a somewhat sleepy, stable play with predictable cash flows and low-but-steady growth. However, the surge in demand for Compute has broadened the scope of a data center from a play on physical real estate to a play on digital real estate as well. Data centers can now optimize on cost (power price), location, efficiency, uptime, and hardware profile (GPUs, FPGAs, ASICs, etc.) to compete for contracts from hyperscalers and new AI players.​
• Data centers, while foundational infrastructure players that are generally abstracted away in the Compute Stack, are complex operations. The location of data centers is important (close to power sources, close to exchanges for high frequency trading / minimal latency, etc.). Data centers need to be increasingly energy efficient. Lastly, security is paramount, with the need for sacrosanct data privacy and computational integrity. ​

The Compute Stack - Hardware Players

• Although the hardware space has been dominated by the meteoric rise of NVIDIA in 2023, the hardware landscape almost certainly will diversify over time (either by regulatory demand or by market demand), as we could see an arms race in hardware solutions tailored for AI. ​
• Currently, GPUs are used to train LLMs in AI, with NVIDIA’s A100 and H100s as the most popular and most high-profile solutions today. It is unlikely that NVIDIA’s A100s and H100s will be the only solutions for AI training and inference going forward, and other hardware players and solutions are likely to come into the fray.​
• It remains to be seen whether more specialized hardware in the form of ASICs or FPGAs can be used for more efficient AI utility, just as the Bitcoin mining industry evolved from CPUs to GPUs to specialized ASICs. However, for now, NVIDIA’s GPUs are the gold standard.​

The Compute Stack - Cloud Service Providers / Hyperscalers

• The Compute and AI revolution is manifesting itself in a top-down, trickle-down fashion. Cloud Service Providers (CSPs) are not only the “front ends” for AI and other Compute applications, but the concentration of Cloud Service Providers marketshare into the major hyperscalers has resulted in vertical integration potential across the Compute Stack.​
• The largest hyperscalers – namely AWS, Google Cloud, and Microsoft Azure – have built massive economies of scale, brand recognition, regulatory and trust infrastructure, and adoption ahead of the surge in AI. As a result, the Cloud Service Provider space is top-heavy, and the major AI front ends are powered by the hyperscalers (ChatGPT via Microsoft/OpenAI, Google Bard).​
• The overarching theme of this piece is that incumbents will see increasing competition from new entrants across the Compute Stack, and we expect the same for the hyperscalers as the CSP landscape decentralizes.

The Compute Stack - Power Producers

• Last, the underlying power producers are used by all other parts of the Compute Stack, chiefly data centers and hyperscalers. ​
• While power consumption of the AI and Compute industry has not yet entered mainstream discussion (analogous to how the BTC mining industry became scrutinized for its power consumption – and thus pivoted to add renewable, ”green” mining solutions – it is inevitable that the resource consumption of the exponentially growing AI sector will enter the spotlight.​
• Therefore, power players will be of increasing importance in the Compute Stack as an emphasis on clean, renewable power will be preferred. Power producers that proactively look to onboard Compute users and seek to capture this growing market could emerge as winners.​
• Ultimately, the Compute Stack will be democratized as it matures over time.​

‍

‍