AI’s Hidden Cost Structure: How Compute Spending Affects Gross Margins

AI’s Hidden Cost Structure: How Compute Spending Affects Gross Margins

For most traditional software businesses, more customer usage has generally added little cost. Once the product is built, serving an additional user or request typically costs very little, so a large share of each new dollar of revenue can flow through to profit. For many AI products, that relationship can work differently. Each time a customer uses the product, the company pays for the compute needed to run the underlying model, so costs can rise along with usage. As a result, how these companies pay to run their models, and how they charge customers in return, can have a meaningful effect on their margins. In this blog, learn more about how compute spending can shape the cost structure of AI companies, how different AI business models and pricing approaches respond to that cost, and why success can sometimes make the math harder.

Where the Money Goes

For AI companies, margins have become a focal point in a way they rarely were for traditional software. Gross margin generally refers to the share of revenue a company keeps after paying the direct costs of delivering its product, often called cost of goods sold, or COGS. For traditional software, those direct costs have typically been low, which is part of why software has long been associated with high margins.

For many AI companies, a large piece of COGS comes from compute, meaning the processing power needed to build and run the underlying models. Compute generally splits into two buckets. Training is the upfront cost of building or improving a model, while inference is the cost of running that model each time a customer uses it. Industry estimates have put inference at the large majority of a model’s lifetime compute cost, with training a much smaller share. That distinction matters, because inference is charged with use. The more a product gets used, the more it can cost to run. This differs from traditional software, where most of the cost is incurred up front and serving one more customer adds little.

The Margins, by the Numbers

The gap shows up clearly in benchmark data. Traditional software businesses have generally run at gross margins in the range of 70% to 85%, while AI-native companies have been reported closer to 45% to 55%.[i] To make it concrete, one analysis walked through adding an AI assistant to a roughly $80 software seat: the inference and supporting infrastructure could add around $15 of direct variable cost, pulling that seat’s gross margin from about 80% down toward 65%.[ii]

The picture is not fixed, though. Survey data has shown average AI gross margins improving year over year, from roughly 41% in 2024 to about 45% in 2025 and around 52% in 2026, as companies optimize how they route and price their models. For anyone assessing technology startups, that can make a single snapshot harder to read, since a low margin today may reflect a young pricing model rather than a permanent ceiling. Margins may also look quite different from early-stage to late-stage as a company matures.[iii]

How AI Companies Charge for Their Products

Because compute costs can scale with usage, the pricing model an AI company chooses is closely tied to its margins. Several approaches have emerged, each attempting to balance predictability, customer preference, and margins differently.

Per-Seat and Flat Subscription Pricing

The familiar approach charges a fixed fee per user or a flat monthly rate. It is easy to budget for and simple to sell. The drawback for AI products is that usage can vary widely within one tier. A heavy user running thousands of requests a day may cost far more to serve than a colleague who logs in occasionally, yet both may pay the same, so a flat plan can quietly subsidize the heaviest users.

Usage-Based Pricing

Usage-based pricing charges according to how much a customer consumes, often measured in tokens, the units of text a model processes. This can align revenue more closely with the cost of serving each customer, which may help protect margins. The trade-off is predictability, since bills can fluctuate month to month. Model providers such as OpenAI and Anthropic have generally priced their interfaces per token rather than a flat fee.

Outcome-Based and Agentic Pricing

A newer approach charges for results rather than access or consumption. Instead of paying per user or per token, a customer might pay per task completed or problem solved. In customer support, Intercom’s Fin has reportedly charged per issue resolved, and Salesforce’s Agentforce has reportedly priced around a set amount per conversation.[iv] This can align a vendor’s incentives with the value delivered, though measuring and attributing an outcome is complex and revenue can be hard to forecast.

Hybrid Pricing

Many companies have landed in between, pairing a fixed base fee with variable charges. A hybrid plan might include a set number of tasks each month for a base price, then charge per task beyond that. This can provide a predictable revenue floor while still capturing cost when usage runs high, and is sometimes described as a bridge between subscriptions and outcome-based models.

When Usage Outpaced Pricing

Few examples illustrate this better than Cursor, an AI coding tool. For much of 2024 and early 2025, Cursor offered a flat $20 per month plan that effectively subsidized premium model usage, with annual recurring revenue (ARR) reportedly climbing from near zero to around $100 million within roughly a year. But the flat structure proved hard to sustain as more users leaned on expensive frontier models. In mid-2025, Cursor shifted to usage-based pricing, and the rollout drew backlash over surprise charges. The company’s CEO publicly apologized in July 2025, acknowledged the communication had been unclear, and offered refunds for unexpected overage charges.[v]

Cursor is not alone. According to industry coverage, Replit, a browser-based development platform, reportedly saw its gross margin fall under 10% and even turn negative during a usage surge in 2024. Pricing changes brought it back into a 20% to 30% range, though still well below the generally accepted industry benchmark of 70% to 85% margins for a software company.[vi]

It’s Not Just Startups

The same pressure is visible among larger, established players. GitHub Copilot has reportedly moved toward usage-based billing, a change some read as a signal that flat AI subscriptions come under strain when heavy users consume outsized amounts of compute.[vii]

Even incumbents with strong margins have felt a nudge. HubSpot’s gross margin reportedly slipped modestly, from around 85% in 2024 to roughly 84% by early 2026, a move its finance team has attributed partly to the cost of rolling out AI across its platform, according to coverage of its earnings. The shift is small, but the direction points the same way as the startup examples.[viii]

What This Means for Investors

For investors evaluating technology startups, an AI company’s cost structure and pricing model can be as informative as its growth rate. Rapid growth built on a flat plan that subsidizes heavy usage may look different from growth supported by pricing that covers the cost of each additional user. Cursor is a useful example. Its revenue reportedly climbed quickly under a flat plan, but the same pricing that helped fuel that growth left it exposed as more users leaned on expensive models.

Margins may also need to be read in context. A gross margin below the 70% to 85% range typical of traditional software is not necessarily a warning sign for an AI company, particularly at an early stage. The trend over time may say more, along with whether the pricing model lets revenue rise alongside the cost of serving the heaviest users. During due diligence, investors may want to consider whether a company’s unit economics appear durable or dependent on temporary subsidies.

Final Thoughts

Compute is a real and recurring cost for AI companies, and because much of it scales with usage, each additional request can add to a company’s costs. That differs from traditional software, where serving more usage has typically added little to the cost of delivering the product. The pricing model a company chooses, whether per-seat, usage-based, outcome-based, or hybrid, can shape how well its revenue keeps pace with that cost. That strain has shown up in gross margins across the spectrum, from fast-growing startups like Cursor to established platforms layering AI onto existing products. As pricing models and hardware continue to evolve, these margins may keep shifting, which can make them a useful thing to watch when trying to understand how an AI business really works.

Are you ready to invest in startups? Sign up for a MicroVentures account to start investing!

Want to learn more about investing in startups? Check out the following MicroVentures blogs to learn more:

• Powering the Future: Investing in Artificial Intelligence Startups
• Hardware vs. Software Startups: Key Differences Investors Should Understand
• Understanding Startup Revenue Models
• The Energy Bottleneck: How Data Center Power Constraints May Reshape AI Investing
• Following the Money: Strategic Investors in the AI Lab Race