Key Takeaways
- ICONIQ's January 2026 State of AI puts average gross margin on AI products at 41% in 2024, 45% in 2025 and a projected 52% in 2026 — against reported subscription gross margins of 80.0% at ServiceNow, 82.7% at Salesforce and 91.1% at Adobe.
- Microsoft's FY2025 10-K states that Microsoft Cloud gross margin "decreased to 69% driven by the impact of scaling our AI infrastructure".
- Model inference is 23% of the AI product cost stack at scaling companies, up from 20% pre-launch, while talent's share falls from 32% to 26%.
- A business at 52% gross margin needs 1.6 times the revenue of one at 85% to fund the same absolute gross profit.
Software used to have no cost of goods sold
The valuation architecture of the software industry rests on one property: the marginal cost of serving another customer is zero. Copy the bits, collect the fee. That property produced gross margins in the eighties, and above ninety at the strongest, and it justifies paying a multiple of revenue rather than of earnings: at those levels, revenue and gross profit are nearly the same thing.
Inference breaks the property. Every query consumes compute that somebody pays for, and the cost recurs each time the feature is used. The chart above sets both structures on one axis.
The numbers, and what they are not
ICONIQ's January 2026 State of AI, drawn from a survey of 269 companies, reports average gross margin on AI products of 41% in 2024, 45% in 2025 and a projected 52% for 2026.
Two clarifications matter, because both have been misreported. The 52% is a projection, not a reported result, and it is the margin on AI products specifically, not company-wide gross margin at AI-native firms. And the widely circulated claim that inference consumes 23% of revenue is wrong. The 23% is inference's share of the AI product cost stack, alongside talent at 26%, infrastructure and cloud at 17%, and data storage at 14%. Those are different denominators, and the difference changes the conclusion.
What the cost stack does show is a shift in composition. As products move from pre-launch to scaling, model inference rises from 20% to 23% of cost while talent falls from 32% to 26%. The cost of software is migrating from people, a fixed cost, to compute, which is not.
The comparison that does the work is in the filings
ICONIQ does not publish a benchmark for traditional software margins, and the 80-90% figure often attributed to it does not appear in the report. It does not need to: the comparison is in audited filings, a stronger source.
For their most recent fiscal years: Adobe's subscription gross margin was 91.1%; Salesforce's subscription and support 82.7%; ServiceNow's subscription 80.0%; Datadog's total 80.0%; Palantir's 82.4%. Snowflake, compute-intensive by construction, reported a GAAP product gross margin of 72%. Against that range, a projected 52% is not a rounding difference. It is a different business model wearing the same label.
Microsoft has already said this out loud
The strongest evidence is not a survey but a disclosure in an audited annual report. Microsoft's FY2025 Form 10-K: "Microsoft Cloud gross margin percentage decreased to 69% driven by the impact of scaling our AI infrastructure, offset in part by efficiency gains in Azure."
And it has kept falling. The 69% is the FY2025 annual figure; on the FY26 Q3 earnings call, CFO Amy Hood said Microsoft Cloud gross margin percentage "was slightly better than expected at 66%, and down year-over-year due to continued investments in AI", and guided the following quarter to "roughly 64%". Snowflake supplies the other tell: it guided non-GAAP product gross margin down to 75.0% for FY2027, from 75.8%. Companies do not guide gross margin lower for fun, and the infrastructure spending driving it is still accelerating.
Falling inference prices have not fixed it, and that is the puzzle
The obvious objection is that inference is getting cheaper fast, so the margin problem solves itself. The price data is dramatic. Stanford's AI Index 2025 reports that the inference cost of a model scoring GPT-3.5's level on MMLU fell "from $20 per million tokens in November 2022 to just $0.07 per million tokens by October 2024 (Gemini-1.5-Flash-8B) — a more than 280-fold reduction in approximately 1.5 years." The series is historical: it ends in October 2024, and the Index's 2026 edition does not extend it.
And yet AI product gross margins are 52%, not 85%. A 280-fold decline in unit cost has not restored software economics, because falling unit costs are being spent rather than banked: cheaper tokens make reasoning models, agents and always-on features viable, and each consumes far more tokens per task. That is why the margin line has crept from 41% to 45% to a projected 52%, rather than snapping back.
What this cannot settle
The ICONIQ figure is self-reported survey data from private companies, with a shifting sample, and the 2026 number is a projection made by the same firms being measured. It is not audited, and not equivalent to the SEC-filed figures it is compared against. The chart's comparison is deliberately unlike-for-like — AI product margin against company-level subscription margin — because no company reports an audited "AI gross margin" line.
The thesis would be falsified if AI product margins climbed past 70% while token consumption per task stabilised, showing the marginal cost is returning to zero after all. It would also weaken if the compute cost proves a transitional subsidy: much inference runs on hardware sold at high margin, and a normalisation in accelerator pricing would flow straight into software gross margins.
What a 52% gross margin actually costs an equity holder
A company at 52% gross margin must generate 1.6 times the revenue of one at 85% to produce the same gross profit. Put the other way, an AI-native business must strip roughly 33 points of revenue out of its operating cost base to match an incumbent's operating margin — or charge more, or reach a scale the incumbent never needed.
Software was rerated in the 2010s on the premise that revenue converts almost entirely to gross profit. For the AI cohort, roughly half of it does. Whether the market is still paying incumbent multiples for a cohort with service-business unit economics is not a forecast — it is an arithmetic check available in every filing, and one the capital cycle upstream of these companies makes more pressing, not less.
This content is for informational purposes only and does not constitute financial advice. Always do your own research or consult a qualified financial adviser before making investment decisions.