--- title: "Understanding Quantization and ollama model storage" description: "This blog clarify the knowledge of what is Quantization and what is storage models in ollama." author: "Yash Srivastava" published: 2026-06-06 updated: 2026-06-08 canonical: https://answers.mindstick.com/blog/364/understanding-quantization-and-ollama-model-storage category: "technology" tags: ["ollama", "ai model", "llm", "quantization"] reading_time: 4 minutes --- # Understanding Quantization and ollama model storage ## What is Quantization? Quantization is a technique which is used to reduce the size of AI models such that the [AI model](https://www.mindstick.com/news/4386/tencent-unveils-new-ai-model-claims-it-responds-faster-than-deepseek-r1) can also run on normal computers or laptops. In simple words we can say that Quantization is process to make AI model less in size and lightweight. In AI models there are billions of [parameters](https://www.mindstick.com/articles/87/passing-parameters-in-c-sharp) and numerical values stored which is normally stored in high precision which causes the large size of AI model which requires more RAM and powerful [hardware](https://www.mindstick.com/blog/303100/best-pc-hardware-2023) to run. Quantization reduces the precision of the values which reduces the size of the AI model and also reduces the requirement of hardware. *Example:* suppose any AI model stores the value like this: ```plaintext 0.987654321 ``` After Quantization the value is change as something like this: ```plaintext 0.99 ``` It caused some precision loss but the storage an memory usage is significantly reduced. That's why Quantization model use less storage, consumes less RAM, Load faster and also run on normal laptops. ### Why do we need Quantization? Original AI models are very large in size and to run these AI model highly powerful hardware are required. *Example:* ```plaintext Original Model ↓ Large Storage Requirement ↓ More RAM Usage ↓ High Hardware Requirement ``` These models can not run in each system. So to solve these problem Quantization is used. ### ### Common Quantization Types When you [download](https://www.mindstick.com/articles/188403/website-to-download-photos-from-instagram) ollama oy any AI model then you may introduce by some terms like: ```plaintext Q2 Q3 Q4 Q5 Q6 Q8 ``` this represents the Quantization level of the model. In simple words It tells about the how the precision value is stored of any model. Generally, Quantization level defines the size of model but differ can be visible in terms of accuracy and response. ## Q2 Quantization Its an very aggressive quantization, these are the very small in size models, They consumes the minimum RAM, There can be noticeable reduction in Accuracy. Q2 Quantization is Suitable for: - Very low-end systems - Testing purposes ## Q4 Quantization This is the one of the popular Quantization level. It maintains good balance between quality and performance, They consumes less RAM and faster inference. Many ollama models are available in Q4 in default. Q4 Quantization is Suitable for: - Most [beginners](https://www.mindstick.com/blog/300589/the-basics-of-seo-for-beginners) - Normal laptops - Everyday usage ## Q5 Quantization It provides the balance between Q4 and Q8 Quantization. It provides better accuracy than Q4, usage manageable hardware requirements. Q5 Quantization is Suitable for: - Users who want slightly better quality - Mid-range systems ## Q6 Quantization Q6 provides higher precision, Better response quality but it consumes more RAM in compare of Q4 and Q5. Q6 Quantization is Suitable for: - Advanced users - Systems with sufficient RAM ## Q8 Quantization It is one of the highest commonly used quantization levels. It provides Better response quality, more accurate outputs and these are closer to the original model. but these are large size model and it requires higher RAM than other quantization models. Q8 Quantization is Suitable for: - Powerful systems - Users who prioritize quality over speed ***[Comparison](https://www.mindstick.com/interview/785/what-are-the-limitations-of-mysql-in-comparison-of-oracle) Table*** | Quantization | Model Size | RAM Usage | Speed | Accuracy | | --- | --- | --- | --- | --- | | Q2 | Very Small | Very Low | Very Fast | Low | | Q4 | Small | Low | Fast | Good | | Q5 | Medium | Medium | Good | Better | | Q6 | Larger | Higher | Moderate | Very Good | | Q8 | Largest | Highest | Slower | Best | ### Understanding Ollama Model storage now we have to understand where the model originally gets stored when we download the model. when we run the command ```plaintext ollama pull llama3 ``` then ollama download the model from [internet](https://www.mindstick.com/articles/12337/no-one-can-realize-the-dignity-of-the-internet-of-things) and stores in local system. ### Where Does Ollama Store Models? #### Windows ```plaintext C:\Users\\.ollama\models ``` #### Linux ```plaintext ~/.ollama/models ``` #### macOS ```plaintext ~/.ollama/models ``` these location are ollama storage folder. ## ### Understanding the Folder Structure Models folder is consist my mainly two important folders which are: ```plaintext models │ ├── blobs ├── manifests ``` ## What is the Blobs Folder? Blobs folder stores the actual data of the model. This folder is consist of AI model files. These are the normal model data files. ## What is the Manifest Folder? This folder stores the information of the model like model name, version, configuration, tags, etc. Understanding Storage is important because it can full the storage of c drive when [multiple](https://www.mindstick.com/forum/2323/handling-multiple-submit-button-in-asp-dot-net-mvc) highly weighted model are load in any system. read more about ollama: previous topic: [understanding Token, context window and CPU vs GPU in ollama](https://answers.mindstick.com/blog/361/understanding-token-context-window-and-cpu-vs-gpu-in-ollama)\ next topic: [Understanding Modelfiles in Ollama](https://answers.mindstick.com/blog/373/understanding-modelfiles-in-ollama) --- Original Source: https://answers.mindstick.com/blog/364/understanding-quantization-and-ollama-model-storage Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.