Consumption weighted token prices have declined approximately 50% over six months and average token prices have declined 62% at the frontier, reflecting both the growing share of cheaper open-weight models and price cuts by proprietary providers, likely driven by some combination of improved inference efficiency and competitive pressure. Increased competition from open-weight models may exert pressure on margins at the frontier through time, but the expansion of token volume suggests overall inference demand is accelerating.
Open-weight models now account for around 65-70% of smart routing token volume, potentially putting pressure on excess rents at the frontier as cheaper models become “good enough” for a growing range of enterprise tasks, especially given US-developed open-weight models offer greater flexibility around customisation, hosting and data control. Caution is warranted in extrapolating from OpenRouter and Vercel data, since users who self-select into model-routing platforms may be more willing to experiment with and switch between models, exhibiting less inertia than the broader AI customer base; however the direction of travel is nevertheless informative, especially as we expect adoption to become increasingly cost-conscious through time.
The recent stall in total expenditure in Ramp’s monthly spend data provides some further grounds for caution, as it shows essentially no growth in overall enterprise spend in the last two months (Aug and Sept), likely reflecting the interaction of lower token prices with increased volume. The data also implies that 80% of spend on frontier labs is driven by 1% of firms, and that the top 1% spent $9,568 per employee per month in Sept vs the median firm that spent $15.7 per employee per month, illustrating how concentrated spending remains. The recent plateau may prove important as total expenditure determines the revenues available to remunerate the AI investment cycle, and it appears that some combination of lower prices and increased competition from open-source may be causing total spend to flatline. The upcoming October release will be important as more timely token volume data from October imply a further acceleration of inference demand, but the implications for total spend remain unclear given shifts in token prices and composition of spend.
Token volumes have risen approximately 5x over the last 6 months and around 50% over the last four weeks, consistent with a substantial demand response as intelligence becomes cheaper. Where that expansion exceeds improvements in computational efficiency, demand for GPU-hours increases, and GPU rental rates suggest that available compute capacity remains scarce: H100 rentals are up approximately 14% over six months and 7% over the last 30 days, while B200 rentals have risen 27.5% over six months, implying robust demand for compute. Continued strength in older H100s is particularly striking against the depreciation one would ordinarily expect from technological obsolescence, implying sellers of scarce compute are well placed to extract rents.
A firm’s margin on each additional unit of output is the difference between its selling price and marginal cost, so falling output prices and rising input costs squeeze margins unless productivity improves sufficiently to compensate. For the frontier labs, competition from cheaper open-weight models can exert both pressures simultaneously, pushing token prices lower whilst stimulating inference demand that bids up the cost of compute. At unchanged inference efficiency, higher GPU rental rates increase the marginal cost of supplying a token, leaving the frontier receiving less for what it sells whilst paying more to produce it. The question is therefore whether improvements in tokens served per GPU-hour (inference efficiency) can offset both the decline in selling prices and the increase in compute costs in perpetuity. Gross margin can be approximated as follows, assuming compute dominates serving costs and holding workload mix constant:
Using the 27.5% rise in B200 GPU rental rates as an illustrative increase in compute costs, combined with the 50% decline in average token prices requires throughput to increase 2.55-fold, merely to preserve the gross-margin. This is not an unsurmountable issue, as reporting suggests that inference efficiency gains in the last year have been substantial, and compute costs can be locked in via multi year deals. However the marginal economics of inference can still remain a source of pressure, particularly on a forward basis, should open source competition intensify, or should progress on inference efficiency stall. These calculations are illustrative only and intended to highlight the efficiency requirement rather than estimate any individual lab’s margins, since public rental benchmarks differ from contracted compute costs and published token prices differ from realised revenues.
Hence, a clear focus is whether inference efficiency gains can continue to support margins despite intense competition for both market share and compute, particularly as the easier improvements or low-hanging fruit may, through time, become exhausted. New hardware offers greater throughput but also commands higher prices, absorbing some of the incremental benefit. Moreover, as efficiency improves, those same benefits may apply to open-weight models too, such that another round of price cuts may stimulate further inference and renew the pressure on scarce compute, leaving the frontier obliged to deliver further successive gains simply to defend its existing margins. None of this implies that the fundamentals of the business model are broken; however, markets move on gaps between expectations and reality, and we question whether expectations for frontier profitability fully accommodate sustained competition that passes efficiency gains through to customers whilst infrastructure suppliers capture the benefits of scarcity in the form of excess rents.
We consequently see a more nuanced outlook for the AI ecosystem, with strong inference demand favouring suppliers of scarce compute as competition raises questions around excess returns at the frontier. The economics may resemble those of airlines and airports, with frontier labs as the established airlines, providing an essential service to the economy in a highly competitive market and sees open-weight models as the low-cost carriers, expanding the market through cheaper fares whilst constraining what incumbents can charge. The hyperscalers’ infrastructure businesses resemble the airports, with stickier demand and greater scope for higher, more durable margins from supplying scarce compute, and a commercial architecture that is model agnostic in the long run. A reassessment of profitability at the frontier on margin or commoditisation concerns would nevertheless be challenging for the complex as a whole, given how much infrastructure investment rests on future spending commitments. Within that framing, we see the hyperscalers as potentially the most durable beneficiaries of growing inference demand, with diversified cash flows and strong balance sheets allowing them to fund additional capacity and withstand a reassessment of frontier profitability more comfortably than leveraged neoclouds.
Copyright © Citadel Enterprise Americas LLC or one of its affiliates. All rights reserved.
Legal Entities Disseminating this Material: This material is disseminated in the United Kingdom by Citadel Securities (Europe) Limited (“CDGE”) authorized and regulated by the Financial Conduct Authority (“FCA”) (Registered company number: 05462867); in the European Union by Citadel Securities GCS (Ireland) Limited (“CSGI”) and its Paris Branch authorized and regulated by the Central Bank of Ireland (“CBI”) (Registration Number: C173437); in Hong Kong by Citadel Securities (Hong Kong) Limited (“CDHK”) licensed by the Securities and Futures Commission of Hong Kong (“SFC”), in Japan by Citadel Securities Japan Co., Ltd (“CSJC”) registered as a Type 1 financial instruments business operator with the Japan Financial Services Agency (“JFSA”); and in the United States of America by Citadel Securities LLC (“CDRG”) registered with the Securities Exchange Commission (“SEC”), Financial Industry Regulatory Authority (“FINRA”), and Securities Investor Protection Corporation (“SIPC”), Citadel Securities Institutional LLC (“CSIN”) registered with the SEC, FINRA, and SIPC, or Citadel Securities Swap Dealer LLC (“CSSD”) registered with the SEC, Commodities Futures Trading Commission (“CFTC”), and National Futures Association (“NFA”). Unless governing law permits otherwise, you must contact a Citadel Securities entity in your home jurisdiction if you want to use our services in effecting a transaction in any financial instruments or securities, including derivatives.
FOR INSTITUTIONAL USE ONLY; FOR PROFESSIONAL CLIENTS AND ELIGIBLE COUNTERPARTIES ONLY. This material is not intended as and does not constitute investment research. Contents of this material will be strictly limited to non-specific, generic information (i.e. macro events/topics) and are not subject to the Markets in Financial Instruments directive (MiFID II) or FINRA research rules. This material does not constitute an offer, solicitation, invitation, or inducement to purchase, acquire, subscribe to, provide, or sell any financial instrument or otherwise engage in investment activity. Please see additional important disclosures, including disclosures that may be relevant to your country of residence or business at www.citadelsecurities.com/GlobalSalesTrading.
