one million tpus drawing over a gigawatt of power. that's basically a nuclear reactor worth of electricity. for one company. by 2026.
like i don't think people are really connecting what this means. we're not talking about buying more servers. we're talking about the kind of power infrastructure that used to only exist for cities or countries. anthropic alone is gonna need nuclear-plant-scale energy just to train their next models.
and this is where it gets weird..training runs used to take megawatts. now we're at gigawatts. plural, probably, once you factor in openai and google and meta all doing the same thing at the same time. that's an entirely new category of energy demand just appearing in like 2-3 years.
the scaling laws everyone keeps assuming will continue? they assume you can just keep adding compute. but nobody's really pricing in that you're not just building bigger clusters anymore. you're building power plants. or negotiating with governments for grid access. the bottleneck isn't the chips, it's whether the electrical infrastructure can even support this.
and most people are still thinking about this like it's just cloud computing getting bigger. it's not. it's fundamentally different when a single training run needs the power output of a small country.
this stopped being just a tech story a while ago.