Trying out a new format this week. Instead of links, here’s some shorts. Too little for a post, too much for a link. This week we have Power variation in AI data centres Using Claude Code for Terraform Adopting new CLI tools Historical maps Power variation in AI data centres (Link requires registration and full article isn’t free, but there’s a massive preview which is all I have access to) ...
Piotr Mazurek and Felix Gabriel have an amazing post up on LLM Inference Economics from First Principles, which I found on Bluesky. They go into a huge amount of detail about how inference works and how that affects processing speed. But I saw the graph above and thought “we can get energy from that”. And so I asked chatgpt-o3: “Looking at these graphs of throughput at different batch sizes on a 4xH100 80gb cluster, what ranges of power per token do they equate to?” ...
We have some actual numbers on tokens! In their commentary on the latest earnings report, Microsoft report: We processed over 100 trillion tokens this quarter, up 5X year-over-year – including a record 50 trillion tokens last month alone So we’ve got a baseline, and can estimate a growth rate. Assuming exponential growth and a three month token split of 19/31/50 we get 62% month on month growth, or, if it keeps going 100 T → 424 T → 1.8 P → 7.6 P, which is lower than the 5x yoy growth MS quote. ...
As Andy Massick explains well, if you are making a few requests per day to a simple non-reasoning text based AI model the carbon impact is, to use a technical term, bugger all 1. I probably ask ChatGPT and Claude around 8 questions a day on average. Over the course of a year of using them, this uses up the same energy as running a single space heater in my room for 2 hours in total. Not 2 hours per day, just a one-off use of a single space heater for 2 hours […] By being vegan, I have as much climate impact as not prompting ChatGPT 400,000 times each year ...