Fireworks AI says routing between models cuts AI bills
Fireworks AI's Rob Ferguson told The Register that mixing several models and tracking cost per task beats running one model or buying your own GPUs.
The facts
- Fireworks AI VP Rob Ferguson told The Register the firm is steering customers away from on-prem AI, saying 10,000 GPUs represent a billion dollars of capital that should not sit idle.
- Fireworks says it builds custom routers that grade each task's difficulty and send it to the best-suited model, rather than just acting as a basic AI gateway.
- Ferguson claims that mixing closed and open models can cut a customer's bill by at least 50 percent, and that the firm talks of six to eight times cheaper. These are the company's own claims, not independently verified.
- Ferguson said Fireworks cached about 90 percent of tasks for one data science customer, while a look at Anthropic's Opus showed about 30 percent. The source gives no further detail on how this was measured.
Case notes · Danny's take
Why it matters
This is a vendor selling its own model, so read it that way. Still, the point about cost per task is sound. A cheap token price means little if the model keeps redoing work. Ask who benefits when the bill is hard to read. The meter is not the cost. The task is.
For builders
Track cost per task, not price per token, and test whether a cache is actually being hit. Build your own evaluation criteria before you commit to one model. Get the vendor's savings claims checked against your own workload, not their slide.
Not financial advice. Do your own investigation.
This story was summarised with AI from The Register's reporting and checked against our editorial policy. Danny's take reflects his views. Disclaimer.
Danny Walter
Founder of Moon Republic, Volt Terminal and Moon Republic Academy. Author of The Market Is a Crime Scene and After the Applause.