Why Cheaper AI Tokens Led to 13 Times More Spending
Token prices on Vercel's AI Gateway roughly halved, usage rose 26x and spend rose 13x. Harpreet Arora explains what gateway data shows about open-weight and frontier models.
· 4 min read
Harpreet Arora, lead for agentic infrastructure at Vercel · MTS
Harpreet Arora leads agentic infrastructure at Vercel, including its AI Gateway, which routes customers' requests across models from AI labs and open-weight providers. Vercel publishes a monthly production index of what it sees on the gateway. He spoke with MTS live at Vercel's Ship conference.
Summary
Arora says open-weight models passed half of AI Gateway's tokens in August and over a quarter of its dollars in October, a sign enterprises now use them in production. Most of that spend goes to US-based hosts with zero data retention, even when the models come from Chinese labs. Frontier model spend still grew about 10x since December while open-weight spend grew about 70x, because companies explore with frontier models and move repeat work to cheaper ones. With token prices roughly halved and usage up 26x, total spend rose 13x, which he calls Jevons paradox at work. These signals are from an MTS interview with Harpreet Arora.
- Choice Overload. In a single week, Arora says, Google released a new Gemini model and Meta released both an open-weight model and another model. The frontier changes so fast that even people deep in AI struggle to keep up. Customers have plenty of choice, but at some point it overwhelms them and makes infrastructure decisions hard. Vercel built AI Gateway so developers can try the latest models while keeping their infrastructure reliable. ▶ 1:22
Quote: “At some point the choice gets overwhelming. And how do you make infrastructure decisions in a world that's fluid?” (Harpreet Arora, ▶ 2:02)
Spotted an error? Email corrections@signalful.com and we’ll fix it.
