What computer do I need to run AI models locally?

Running AI models on your own computer is possible on a wide range of hardware, but what you can run depends mainly on memory. Here is a practical guide.

The most important factor is memory. The model must fit in memory to run well. That means RAM for CPU-based running, or video memory, called VRAM, on a graphics card for GPU-based running. As a rough guide for 4-bit quantized models:

Small models of 1 to 4 billion parameters need about 2 to 4 GB.
Models of 7 to 9 billion parameters need about 5 to 8 GB.
Models of 12 to 14 billion parameters need about 8 to 12 GB.
Models of 27 to 32 billion parameters need about 18 to 24 GB.
Models of 70 billion parameters need about 40 GB or more.

Add some extra memory for long conversations, because longer context also uses memory.

GPUs make a huge difference to speed. NVIDIA graphics cards are the most widely supported for AI. A card with 8 GB of VRAM can run small models well, 12 to 16 GB handles mid-sized models, and 24 GB opens up larger models. Without a GPU, models still run on the CPU, just more slowly.

Apple Silicon Macs are excellent for local AI. M-series Macs use unified memory, shared between the processor and graphics, so a Mac with 32 GB or more of memory can run surprisingly large models smoothly.

Here are practical setups:

Basic laptop with 8 GB RAM: small models for simple chat, summaries and learning. Expect modest speed.
Laptop or desktop with 16 GB RAM: comfortable for 7 to 8 billion parameter models.
Gaming PC with a 12 to 16 GB NVIDIA GPU: good speed with mid-sized models.
High-end workstation with a 24 GB GPU or Mac with 64 GB or more: larger, much more capable models.
Servers with multiple GPUs: the largest open models, mainly for businesses.

Storage matters too. Model files range from a few gigabytes to over 100 GB. An SSD makes loading faster.

Before buying hardware specifically for local AI, consider the costs. For occasional use, cloud AI services or APIs may be cheaper than an expensive GPU. Local AI makes the most sense when privacy is critical, usage is heavy, offline access matters or you want to learn and experiment.

The simplest first step is to install LM Studio or Ollama on the computer you already have, try a small model and see how it performs before spending money on upgrades.