The big stack game of LLM poker

The next few years is going to make the "$600B Question" look small

I’m sure you read David Cahn’s provocative piece "AI's $600B Question", in which he argues that, given NVDIA’s projected Q4 2024 revenue run rate of $150B, the amount of AI revenue required to payback the enormous investment being made to train and run large language models is now $600B, and we are at least $500B in the hole on that payback. The numbers are certainly staggering… and are just going to get bigger. Until we reach an efficient frontier of the marginal value of adding more compute, or we hit some other roadblock that causes people to lose faith in the current architecture, this is a contest now of “not blinking first”. If you’re a big stack player like META, MSFT, GOOG, or any of the foundation model pure plays, you have no choice but to keep raising your bet — the prize and power of “winning” is too great. If you blink, you are left empty handed, watching someone else count your chips. It’s likely hundreds of billions will be destroyed, and trillions earned. It’s too early to know who the winner or losers are. But for all of us in the startup ecosystem, among many things, it’s going to create new waves of AI opportunities.

Taking a step back, as LLMs progress, they are able to handle more complicated tasks. If today LLMs can handle tasks that would have taken a human thirty minutes to complete, as LLMs progress, they'll be able to handle increasingly complicated tasks that would have taken a human more time. In the next decade, they should be able to handle tasks that would take years for a human to do. Therefore, as the LLMs become more and more sophisticated, the economic value that they will be able to unlock becomes greater and greater.

For example, annually, it is estimated that we spend $1T on software engineers globally. When people talk about GitHub Copilot, you hear people throw around numbers like 10-20% productivity improvements (of course, GitHub claims higher). That translates to $100-200B of value annually were it to be fully deployed (of which GitHub would capture some percentage).

As LLMs progress and are able to go beyond code completion ("copilot") to code authoring ("autopilot"), there is almost no limit in value creation as it would dramatically expand the market – a potential multi-trillion dollar opportunity if someone emerges a dominant player. And that's just coding. We've all experienced the productivity-improving benefits of LLMs (or been on the receiving end of an automated customer support response). The potential value creation and capture with AI is beyond our existing mental models.

The challenge is the amount of capital required to train each successively more sophisticated LLM increases by an order of magnitude, and once a model is leapfrogged by another, the pricing power of the older model quickly falls to zero. There are now more GPT3.5 equivalents for a developer to choose from than would make sense for them to test. Not surprisingly, when GPT3.5 launched in November 2022 it was head and shoulders ahead of any competitive model and cost $0.0200 for 1000 tokens. It's $0.0005 now – 2.5% of its original pricing in just 1.5 years. I can’t remember another technology that has commoditized as quickly as LLMs. It’s a dynamic that makes it almost impossible to rationalize any ROI at this stage in the game because any investment in a LLM is almost instantly depreciated by the next version. But you can’t really skip a step. You need to go through countless worthless versions to get to the ultimate (the idealized “AGI”).

So you have a bit of a perfect storm:

1. The economic value you are able to unlock as models become more sophisticated should increase significantly with each upgrade of the model. The economic value of AGI is constrained only by our imaginations. 

2. Pricing leverage comes from being a step function ahead of the competition, at least along some dimension. If you fall behind, the value of your model to external customers gets rapidly commoditized (of course, there is still value for your internal use cases).

3. MSFT, GOOG, and META have core businesses that produce fire hydrants of cash, Anthropic has found love with GOOG and AMZN, and OpenAI should continue to be able to raise money from sovereigns that have their own (more physical) fire hydrants of cash. 

The net result is that in the short term, until an efficient frontier is reached on the marginal value of continuing to invest in infrastructure with the existing transformer architecture, or we run out of electricity, or a group pulls ahead with an untouchable lead thanks to some smart algorithmic work, investment in this space by these giants should continue to increase dramatically, and costs necessarily precede revenue. The prize is theoretically so large, and if a clear winner emerges, their market opportunity so uncapped, you have to keep increasing your bet.

We all are massive beneficiaries of this battle playing out. The extreme pace of investment in infrastructure / training / etc, combined with the urgency that only comes from intense competition, is giving us all the gift of an insane pace of innovation with models that are able to handle increasingly complicated tasks at bargain basement prices. Applications that might not be possible today, let alone economic (such as most voice and video applications), will be profitable before we know it. Giddy up!