AI Carbon Footprint: What Training Big Models Really Costs

TL;DR: Training a frontier model is a real but bounded climate hit — GPT-3's training run emitted roughly 552 tonnes of CO₂, about the same as 123 petrol cars driven for a year. The harder problem is inference: every query you send runs the model again, and data centres are on track to rival whole countries for electricity demand. The fix isn't quitting AI — it's picking efficient models, cleaner grids, and providers who publish real numbers.

Here's a number that tends to land with a thud. The training run behind GPT-3 pumped out somewhere near 552 tonnes of CO₂ equivalent — about the same as 123 petrol cars driven for a full year. That's not nothing. But it's also not the whole story, and the whole story is far more interesting.

The AI carbon footprint of one training run

The most-cited figures come from a 2021 analysis that included Google engineers: roughly 1,287 megawatt-hours of electricity and 552 tonnes of CO₂e to train GPT-3. Meta has since gone further and published the receipt for Llama 3.1 405B — about 30.84 million GPU hours on H100s.

Where that electricity comes from matters more than the raw count. The same job on a coal-heavy grid versus a hydro-heavy one can differ by roughly tenfold. Location, not model size, is often the biggest lever.

Water usage is the part nobody mentions

Data centres get thirsty. Google's and Microsoft's disclosed water consumption both climbed sharply in their recent sustainability reports, much of it evaporating in cooling towers. There's an indirect draw too — the power plants making your electricity use water as well.

One estimate put GPT-3's training at around 700,000 litres of freshwater. Take those single-model figures as direction, not gospel (nobody's publishing an actual meter reading), but the trend is real and it's local: a data centre in a drought-prone region is a genuine community issue.

The real problem: inference, not training

Training is a one-time cost. Inference runs forever. Google reported back in 2021 that roughly 60% of its machine learning energy went to inference, not training — and that was before chatbots ate the internet.

The US Lawrence Berkeley National Laboratory put American data centres at about 4.4% of national electricity in 2023, with a plausible range of 6.7%–12% by 2028. Utilities are already delaying coal plant retirements to keep up. That's the part I'd worry about.

What actually helps

• Use smaller models. An 8B-parameter model summarises a document fine. Save the frontier model for the hard stuff.
• Pick your region. Google Cloud and AWS both report carbon data per region. A model in Sweden is a different product from the same model in Virginia.
• Cache and batch. Repeated prompts, bloated system instructions and runaway agent loops all burn tokens for nothing.
• Don't retrain for a rounding error. Fine-tune, distil, or just use a better prompt.

Key Takeaways

• GPT-3's training emitted roughly 552 tonnes CO₂e — measurable, but a one-time cost.
• Grid carbon intensity can swing training emissions by around 10×.
• Inference at scale now dwarfs training for most deployed models.
• Water is the underreported cost, and it's hyper-local.

Frequently Asked Questions