Inference Is the Most Important Market in Software

In short : AI inference is set to surpass the database market to become the most important market in software, reaching ~$350b by 2027. This shifts software unit economics: usage dwarfs platform fees, gross margins compress below classic SaaS norms, and infrastructure becomes the dominant component of COGS.

If a founder built a startup in the database market in the mid-2010s, 2% market share meant the company could IPO. No surprise the database was the most important software category in that era.

Inference is about to surpass it.

In 2025, companies paid about $25b to run AI models. This year the market reaches roughly $130b, within striking distance of Gartner’s $161b forecast for databases.¹ By 2027 AI inference reaches ~$350b, passing databases ($190b) by nearly 2x.

That crossover makes inference the most important market in software : a structural transformation mutating every application into an inference reseller.

That shift rewires application/harness unit economics in three ways :

1. Usage could dwarf platform fees & fuel record-setting growth. The inference consumption bill will exceed the seat license or base subscription. This changes AE compensation plans, demands predictable plans for customers who fear blown budgets, & complicates forecasting.

2. Gross margins compress below SaaS norms. The blended 72% gross margin of classic software² will decay unless an application develops a proprietary harness to aggressively compress token overhead,³ or fierce model competition drives inference costs down faster than limited supply gooses prices. This upends CAC & payback economics.

3. Customers who bring their own keys (BYOK) force vendors to trade revenue for gross margin. When an enterprise supplies its own GPU cluster or model API credentials, the software vendor books almost pure software margin, but on a vastly smaller top-line contract.

Could the AI price war prevent some of these changes? Capability-adjusted token costs fall ~47% each quarter.⁴ But list prices for frontier flagships keep resetting upward as capability leaps : the frontier blended price rose from $0.96 to $11.25 in 18 months before GPT-6 Sol reset the tier to $4.00.

These economics matter tremendously to the application layer. Infrastructure costs are suddenly a significant contributor to overall COGS, & potentially more than half of revenue is allocated to it.

Just like a long-haul trucker watching diesel surge beyond $8 per gallon & wondering about their economics, software companies will watch price per token & hunt for efficiencies on their rigs with smaller models, better harnesses, & new routing techniques.

& that creates plenty of opportunity for innovation.

1. https://www.gartner.com/en/documents/7229830 ↩︎

2. https://tomtunguz.com/is-software-profitable/ ↩︎

3. https://tomtunguz.com/the-harness-margin-opportunity/ ↩︎

4. https://epoch.ai/publications/the-plunging-price-of-thought ↩︎

Get the next one in your inbox

The 1-minute read that turns tech data into strategic advantage.
Read by 150k+ founders & operators.

GP at Theory Ventures. Former Google PM. Sharing data-driven insights on AI, web3, & venture capital.

Bloomberg • WSJ • Economist