What 121 Exaflops Looks Like: Google’s TPUs Keep Getting Weirder and Faster

Google’s been making its own chips for over a decade now. Not the kind you’d drop into a gaming rig—these are Tensor Processing Units, or TPUs, and they do one thing: math. A lot of math, very fast.

The idea was simple from the start. AI models need massive matrix multiplications, and general-purpose CPUs or even GPUs aren’t always the most efficient way to do that. So Google designed a chip that basically skips everything except the math. No fancy graphics pipelines, no general-purpose overhead. Just tensor operations, all day.

What started as a niche accelerator for internal workloads has turned into a beast. The latest generation pushes 121 exaflops of compute power. That’s 121 followed by 18 zeros. For context, that’s more raw compute than most supercomputers on the Top500 list combined, and it’s packed into a single TPU pod.

Bandwidth also doubled compared to the previous generation. That matters more than you’d think. AI models aren’t just about peak FLOPs—they need to shuffle weights and activations around constantly. A chip that can compute fast but starves for data is useless. Google clearly learned from past bottlenecks.

I’ve been watching TPU generations roll out since the v2 days, and the jump from v5p to v6 (or whatever they’re calling this one internally) feels bigger than usual. The 121 exaflops figure is higher than I expected. It suggests they’re either using a denser interconnect or they’ve found a way to squeeze more out of the same physical footprint.

Of course, you can’t buy one. TPUs aren’t for sale. They’re tied to Google Cloud and internal products like Search, YouTube, and Gemini. If you want to use them, you rent time. That’s fine for most people, but it does mean you’re locked into Google’s ecosystem. No running TPUs in your home lab.

Still, the numbers speak for themselves. Whether you’re training a massive language model or running real-time inference at scale, TPUs are doing the heavy lifting behind the scenes. They’re weird, proprietary, and insanely fast. I’d love to see a teardown of one, but I’m not holding my breath.

Watch the video Google put together. It’s short, but it gives you a sense of just how many of these chips are packed into a single pod. The scale is ridiculous, and it’s only going to grow.