What is a vector database and do I need one?
1 Answer
A vector database is a database designed to store embeddings, the numerical representations of meaning, and quickly find the items most similar to a query. It is a core component of many AI applications, especially retrieval-augmented generation, semantic search and recommendations.
Why not use a normal database? Traditional databases are excellent at exact matches and filters, such as "find orders where city = 'Kanpur'." But AI applications often need similarity search: "find the 5 document chunks most similar in meaning to this question." Comparing a query against millions of vectors one by one would be slow. Vector databases use special indexing methods, such as approximate nearest neighbour algorithms, to find similar vectors very quickly, even across huge collections.
Popular options include:
1. Dedicated vector databases, such as Pinecone, Weaviate, Qdrant, Milvus and Chroma.
2. Vector extensions for existing databases, such as pgvector for PostgreSQL, and vector search features in MongoDB, Elasticsearch, OpenSearch, Redis and SQL Server.
3. Cloud provider services, such as vector search offerings from AWS, Google Cloud and Azure.
4. Lightweight libraries, such as FAISS, for local or embedded use.
Do you need one? Not always.
You may not need a dedicated vector database if:
Your dataset is small, such as a few thousand chunks. Simple in-memory search or a library like FAISS may be enough.
You already use PostgreSQL or another database with vector support. Adding vectors to your existing database keeps your architecture simple.
Your documents fit within the AI model's context window. With modern long-context models, small document sets can sometimes be included directly in the prompt.
A dedicated vector database makes sense when:
You have millions of vectors.
You need very fast search at high traffic.
You need advanced features like filtering by metadata, hybrid keyword and vector search, or multi-tenant separation.
You want a managed service that handles scaling.
Practical tips:
Store metadata alongside vectors, such as source document, date, category and access permissions, so you can filter results.
Combine vector search with keyword search for better accuracy on names, codes and exact terms.
Enforce access control so users only retrieve documents they are allowed to see.
Plan for updates, so new and changed documents are re-embedded.
For many teams, starting with a vector extension in their existing database is the simplest path, moving to a specialised solution only when scale demands it.