AI & Tech
AI Inference Compute Supply Crunch Far Worse Than Most Realize, CEO Says
No Priors Podcast
Baseten CEO Tuhin Srivastava on the AI Inference Crunch, Custom Models, and Building the Inference Cloud
"There's so much narrative around the supply crunch. And no matter, like, as much as we hear about it, I don't think people realize how bad it really is. Like, there is, you know, there's very, very little Slack compute available."
Despite widespread discussion of AI compute shortages, the reality is significantly more severe according to Base10's CEO. The company operates at mid-90s utilization across 90 clusters in 18 different clouds worldwide, and new capacity contracts now require 3-5 year commitments with 20-30% prepayment for premium GPUs.