The AI stack in October 2026: What each layer is made of, and what limits it

Fig. 1: Conceptual illustration of five satin silver hardware layers on a charcoal drafting grid, representing the layers of the AI stack.

By An (@0xaqt), staff engineer at B3IQ.

Fig. 2: The AI stack in October 2026: What each layer is made of, and what limits it

In short:

• The AI stack is seven lanes in five tiers, each built from the one below: chips, data centers, two kinds of cloud, two kinds of token seller, then agents
• Anthropic's IPO filing, public since 26 October, shows one lab on four chip families, now leasing dedicated accelerators for halls that others build and run
• For 2027 the forecasts we read put the limits in physical things: Micron has sold more than 75% of its 2027 output, and an estimated 14.2 GW of IT power is online at the frontier AI sites Epoch AI tracks, against 38.2 GW planned

Every request to an AI agent ends up as matrix arithmetic on an accelerator, in a building someone had to wire to the grid. We mapped all seven lanes of the AI stack as of 25 October 2026 from more than 800 sources. We also read Anthropic's IPO filing, made public on 26 October, which shows the same layers from one buyer's seat.

Seven lanes, five tiers

Fig. 3: The AI stack in five tiers: silicon, data centers and power, hyperscalers beside neoclouds, model labs beside inference providers, and agents on top

Read it bottom up. Silicon makes the accelerators and the memory stacked beside them. Data centers turn grid power into cooled, powered halls. Two kinds of cloud share the compute tier: hyperscalers such as AWS, Azure, Google Cloud and Oracle, and neoclouds such as CoreWeave, Nebius and Lambda, which mostly rent out whole GPU clusters. Two kinds of token seller share the tier above: model labs that train their own models, and inference providers such as Fireworks and Together that serve open-weight models, much of it on GPUs rented from clouds. Agents sit on top and call models in a loop.

Silicon puts compute beside stacked memory

Fig. 4: One accelerator package: a compute die beside stacks of high-bandwidth memory on a CoWoS base, 72 of them to a rack, and Ethernet switch chips joining the racks

An accelerator is a chip built for the matrix arithmetic that training and running a model needs. A GPU is the general-purpose kind that Nvidia and AMD sell to most buyers. An ASIC is a custom accelerator built for one buyer, such as Amazon's Trainium or Google's TPU, which Google co-designs with Broadcom, MediaTek and now Marvell.

TSMC makes the compute dies, on its N3 and N2 processes for the 2026 and 2027 chips (today's Blackwell is on 4NP). It then mounts each die beside stacks of high-bandwidth memory (HBM), layers of DRAM stacked on top of each other so the processor can read them fast. The packaging step is called CoWoS (chip on wafer on substrate). SK hynix, Samsung and Micron make the HBM. Microsoft's Maia 300, announced on 12 October, carries eight HBM4 stacks per chip. Nvidia sells the result as racks of 72 GPUs, and Broadcom sells the Ethernet switch chips that join tens of thousands of accelerators into one cluster.

A data center sells a megawatt of IT load

Fig. 5: A maximum of 56 MW of gross capacity, 39 MW of contracted IT load, and the grid work that takes longer than the building

The unit a data center sells is a megawatt of critical IT load, the power that reaches the servers. The utility meter reads more, because cooling and electrical losses sit on top: Cipher Mining's Barber Lake expansion, due by January 2027, is contracted for 39 MW of critical IT load, supported by a maximum of 56 MW of additional gross capacity. A landlord hands over either a powered shell, a building with a grid feed where the tenant installs the cooling and electrical plant, or a turnkey hall with that plant finished.

The building is the fast part. A campus phase can go up in 12 to 24 months, but the substation and grid work behind it can take several years longer. EY cites a 48 MW building in Santa Clara that sat as an empty shell in late 2025, waiting on a utility upgrade.

Fig. 6: Meme, the waiting skeleton format: a skeleton in a headset slumps at an office desk with one hand on the mouse, in front of a computer. The caption reads: A 48 MW building in Santa Clara, waiting on a utility upgrade

So builders make power on site with gas turbines and fuel cells, which the industry calls behind the meter. Janus Henderson expects 10.7 GW of the 24.9 GW it sees landing in the US in 2027 to come from behind-the-meter generation.

Two kinds of cloud: a slice of a fleet or the whole server

Fig. 7: Three ways to get a GPU: a hyperscaler sells GPU-hours and model APIs on its own and Nvidia chips, a neocloud rents whole servers in whole clusters, and a marketplace rents other people's GPUs by the second

A hyperscaler runs data centers at gigawatt scale and fills them with Nvidia GPUs and, increasingly, accelerators of its own design. It sells compute two ways: by the GPU-hour, one accelerator for one hour, and by the token through managed model services such as Amazon Bedrock, Google's Gemini Enterprise Agent Platform and Microsoft Foundry. Its own chips are AWS Trainium, Google TPU and Microsoft Maia, and Meta builds the same kind of fleet with its MTIA chips for its own use. Amazon, Microsoft and Alphabet in their July results, and Oracle in September, all said demand is running ahead of the capacity they have.

A neocloud fills a leased building with Nvidia GPUs and rents them out as whole clusters. Many sell what McKinsey calls bare metal as a service. CoreWeave's annual report describes its version as nodes without a hypervisor: whole servers with no virtualization layer. One tier down, GPU marketplaces such as Vast.ai aggregate other people's GPUs and rent them by the second, and RunPod rents by the second too. SemiAnalysis, which tests GPU clouds hands on, says users compare renting a single machine from some marketplaces to playing "Russian Roulette".

Two kinds of token seller: one trains, one serves

Fig. 8: How a serving engine handles requests: many requests batched onto one GPU, a shared prefix read once and reused, and prefill and decode on separate GPUs

A model is an architecture plus its weights, the billions of numbers that encode what it learned. Closed labs such as OpenAI, Anthropic, Google DeepMind and xAI keep their frontier weights private and sell access through their own APIs and the big clouds. Open-weight labs such as DeepSeek, Moonshot, Alibaba's Qwen team and Zhipu publish weights that anyone can host. OpenAI and Anthropic have mostly rented their compute from hyperscalers and neoclouds, and Anthropic now also leases dedicated accelerators. Google DeepMind runs on Google's own TPUs. xAI built SpaceX's Colossus, and since May 2026 SpaceX has also rented it to Anthropic.

Inference is running a trained model to answer a request. A provider loads a model onto GPUs, keeps it warm and sells its output by the token, roughly three quarters of an English word. The open-weight specialists run a serving engine such as vLLM or SGLang, software that batches many requests onto one GPU and caches shared prefixes. A request that starts like an earlier one skips the work on the shared part. DeepSeek splits each request across machines. By its own disclosure, its DeepInfer serving system was measured on 4 GPUs for prefill, reading the prompt, and 64 for decode, writing the answer one token at a time. The vLLM team reports decode throughput within 6% of DeepInfer's published results on one matched GB200 setup. Custom-silicon clouds such as Cerebras serve the same open models at 1,000 or more tokens a second on chips they designed.

An agent is a model in a loop

Fig. 9: An agent loop: a goal goes to the model, the model calls a tool through MCP, the result comes back, and the harness runs the loop until the task is done

An agent is software that uses a model in a loop. It reads a goal, calls a tool such as a terminal, a browser or a database, checks the result and picks the next step until the task is done. Anthropic's working definition is "LLMs using tools based on environmental feedback in a loop". The harness is the code around the model that runs the loop: tool calls, context and permissions. MCP, the Model Context Protocol, is an open standard for how an agent finds and calls tools. Anthropic released it in November 2024 and gave it to the Linux Foundation's Agentic AI Foundation in December 2025.

Agents lean hard on the prefix cache from the tier below: cached prompts were just under 90% of agentic tokens on OpenRouter in September.

Fig. 10: Meme, the Toy Story everywhere format: Buzz Lightyear, the word Lightyear on his chest plate, puts an arm around a worried Woody and points across the room. The caption reads: Cached prompts, just under 90% of agentic tokens on OpenRouter in September

Some agent makers now train on top of open weights too. Cursor's Composer 2 is built on Moonshot's Kimi K2.5.

One IPO filing shows every layer at once

Fig. 11: Anthropic's compute from its S-1: four chip families under one orchestration layer, and Anthropic Direct, which we read as four separate pieces: chips, building, power and operations

Anthropic filed its prospectus publicly on 26 October and has applied to list on the Nasdaq Global Select Market under the symbol ANT.

At the silicon layer it draws on four chip families: AWS Trainium, Google TPUs through Google and Broadcom, Nvidia GPUs including Grace Blackwell and Vera Rubin systems, and AMD Instinct MI450 GPUs in Helios rack-scale systems. The Nvidia capacity comes through Microsoft, SpaceXAI, Nscale, Lambda and other cloud providers, so both kinds of cloud are in it too. A proprietary orchestration layer tests and measures each platform against Anthropic's actual workloads, and the filing says its engineers now use Claude to bring a new chip platform into production faster than the many months it once took.

At the data center layer it is changing how it buys. Under what it calls Anthropic Direct, it leases accelerators, CPUs and HBM through equipment leases and runs them in dedicated facilities leased and operated on its behalf, to its specification. Its first such facility became operational this year. In some of these deals, partners such as Fluidstack procure the land, power and building, and the power comes from the utility or, in some cases, from generation behind the meter. Data centers take longer to secure but cost less than the chips inside them, so Anthropic can lock in a building early and commit to accelerators once it knows what it needs.

Memory and packaging are the bottleneck

Fig. 12: What caps accelerator supply in 2027: more than 75% of Micron's 2027 output sold, HBM4 qualified at all three memory makers, CoWoS capacity estimates from 2025 to 2027, and four 2027 roadmaps that draw on the same three memory makers

The forecasts we read point to one physical limit for 2027: the stacked memory, and the packaging step that mounts it. Micron has sold more than 75% of its 2027 output, and HBM4 for Nvidia's next platform, Rubin, is qualified at all three memory makers, by Jensen Huang's account. TSMC publishes no CoWoS capacity figure. Analysts put it at about 70,000 wafers a month in 2025, 120,000 to 140,000 by the end of 2026, and, in the most recent estimates, 185,000 to 200,000 by the end of 2027.

Nvidia's Rubin and AMD's MI450 need HBM4, and Google's TPU 8 and Amazon's Trainium3 use HBM3E, so these four 2027 roadmaps all draw on the same three memory makers. Digitimes lists T-Glass, ABF substrates, high-layer circuit boards and HBM as the 2027 supply bottlenecks. So if those forecasts hold, components led by memory, not orders, set how many accelerators ship in 2027.

Power sets when the big campuses reach full size

Fig. 13: An estimated 14.2 GW of IT power online at the frontier AI sites Epoch AI tracks, against 38.2 GW planned, and most planned gigawatt sites reaching full power in 2028

As of 26 October, an estimated 14.2 GW of IT power is online across the 95 frontier AI sites in Epoch AI's register, against 38.2 GW planned, by Compute Terminal's sums. Vacancy in the main North American markets was 1.4% at mid-2026, and more than 80% of the 7,481 MW under construction is already leased. Seven of the ten sites in the register that plan 1 GW or more reach full planned power in 2028, though Anthropic and Amazon's New Carlisle site already has about 1 GW online, and Epoch AI projected in April that five of Stargate's six US sites beyond Abilene finish in Q4 2028. Meta's Hyperion gas plants are due at the end of 2028 and the campus by 2030, by local reports. So by the forecasts we read, 2027 gets built from mid-size sites with gas turbines and fuel cells on site, and 2028 is the first year new campuses, full Rubin volume and normal memory supply could arrive together.

From the grid to the agent loop

We set out to draw the whole stack on one page. Grid power becomes IT load, IT load runs accelerators, accelerators become GPU-hours, GPU-hours become tokens, and agents spend tokens in loops. Every layer in this map comes down to hardware that someone has to build, power and keep running, and by the forecasts we read, the limits for 2027 are physical: stacked memory, one packaging step and the grid. Software moves fast on top of that. The machines underneath move at the speed of fabs and substations.

• An accelerator puts compute dies beside stacks of HBM on a CoWoS package, and four 2027 roadmaps draw on the same three memory makers
• Less power reaches the servers than a site is built for: Cipher's Barber Lake expansion is contracted for 39 MW of IT load on a maximum of 56 MW of gross capacity, due by January 2027
• An estimated 14.2 GW of IT power is online at the 95 frontier AI sites Epoch AI tracks, against 38.2 GW planned, and seven of the ten planned gigawatt sites reach full power in 2028
• Many neoclouds rent whole servers with no virtualization layer, and marketplaces rent other people's GPUs by the second
• Serving engines batch many requests onto one GPU and reuse shared prefixes, and cached prompts were just under 90% of agentic tokens on OpenRouter in September
• Micron has sold more than 75% of its 2027 output, so if the forecasts hold, components led by memory, not orders, set how many accelerators ship in 2027

Conclusion

If memory and power are what limit 2027, the next question is who holds the hardware. Across most of this map, GPUs sit in someone else's building, and the people who use them rent time, by the hour or on multi-year contracts. One of the biggest buyers is moving the other way: Anthropic now leases dedicated accelerators and has partners run the building, the power and the operations around them.

We think every team that runs real workloads should get a version of that deal, at any size, and go one step further: own the GPUs, and still have someone run the boring parts. That is b3iq. You own a bare-metal box, we host and operate it, and your workloads stay yours.

Fig. 14: Bao, the most heads-down member of our team, resting on the top stair

By the forecasts we read, memory and power stay tight through 2027. Rain or shine, we stay heads down to deliver the best GPU stack on the planet. If you want one, b3iq.org.

---

Originally published by An (@0xaqt) on X on October 26, 2026.