Why does AI need GPUs?

Asked 15 hours ago Updated 15 hours ago 31 views

1 Answer


0

Graphics processing units, or GPUs, are the main hardware behind modern AI. Companies spend enormous sums on them, and NVIDIA, the leading GPU maker, became one of the world's most valuable companies largely because of AI demand. Here is why.

AI is mostly a huge amount of simple maths. Neural networks perform enormous numbers of multiplications and additions, especially operations on large grids of numbers called matrices. Training a large language model requires an astronomical number of these calculations, and running it for millions of users requires many more.

The difference between CPUs and GPUs:

A CPU, the main processor in a computer, has a relatively small number of powerful cores designed to handle complex tasks one after another very quickly. It is like a few expert professors solving problems sequentially.

A GPU has thousands of smaller cores designed to perform many simple calculations at the same time. It is like thousands of students each solving a simple sum simultaneously.

GPUs were originally designed for graphics in video games, where millions of pixels must be calculated at once. Researchers discovered that the same parallel design was perfect for neural network calculations. A breakthrough moment came in 2012 when a neural network trained on GPUs, called AlexNet, dramatically won an image recognition competition.

Why is memory important too? AI models must keep billions of parameters in fast memory while computing. Data centre GPUs include large amounts of very fast memory, called high-bandwidth memory, and fast connections between GPUs, so many chips can work together on one model.

Other AI chips:

Google's TPUs, tensor processing units, designed specifically for AI and used for training and running Gemini.
AI accelerators from Amazon, Microsoft, Meta and other companies developing their own chips.
AMD's data centre GPUs, competing with NVIDIA.
Neural processing units in phones and laptops, designed for efficient on-device AI.

Why are GPUs so scarce and expensive? Demand from AI companies, cloud providers and governments has surged far beyond supply. Building advanced chips requires specialised factories, and governments have also imposed export controls on advanced AI chips for strategic reasons.

What does this mean for you? Cloud AI services run on GPU data centres, which is why advanced AI usage has costs and limits. Running AI models locally works much better with a good GPU or an Apple Silicon Mac. In India, the IndiaAI Mission has been building shared GPU capacity so startups and researchers can access computing at subsidised rates.

Write Your Answer