Why does AI need so many GPUs and so much electricity?

GPUs (graphics processing units) were designed to draw video game graphics. That job needs doing many simple calculations at the same time. Training AI needs exactly the same thing, billions of multiplications in parallel, so GPUs turned out to be ideal. Nvidia dominates this market, which is why it became one of the world's most valuable companies.

Why so many? Training a top large language model can take tens of thousands of GPUs running for weeks or months. Then, every time millions of people use a chatbot, more computing is needed to answer them.

Why so much electricity? Thousands of chips running non-stop use a lot of power, and they produce heat that needs cooling, often with large amounts of water. Global electricity use by data centres is rising quickly, and the International Energy Agency expects it to roughly double by 2030.

What this means:

• Tech companies are investing in renewable energy and even nuclear power for data centres.
• Countries are building data centres. India is a growing hub, with large projects in Mumbai, Chennai, Hyderabad and elsewhere.
• Chip access has become a geopolitical issue, with export controls on advanced AI chips.

India's IndiaAI Mission provides shared GPU capacity for startups that cannot afford their own.