Why ordinary search is not enough for AI applications
A keyword search finds documents that contain the same words as the question. People rarely use the same words as the document, though. "How do I get my money back?" should match a page titled "Refund policy", even though no word is shared.
Embeddings solve this. An embedding model turns each piece of text into a vector, so texts with similar meaning end up close together. A vector database is built to store millions of these vectors and return the nearest ones in milliseconds.
How a vector database fits into a RAG pipeline
The same five steps sit behind most document chatbots and AI assistants.
Chunk the documents
Split long documents into passages of a few hundred words, often with a small overlap so no idea is cut in half.
Create embeddings
Send each chunk to an embedding model and get back a vector that represents its meaning.
Store vectors with metadata
Save each vector in the vector database with its text and details such as source, date and access rights.
Retrieve on each question
Embed the user's question the same way and ask the database for the top few most similar chunks, optionally filtered by metadata.
Generate the answer
Put the retrieved chunks into the prompt and ask the language model to answer only from them, ideally with citations.
Ideas you need to understand vector search
A few terms come up in every vector database guide and interview.
Embedding
A fixed-length list of numbers produced by a model to represent meaning. Similar meanings give nearby vectors.
Similarity metric
How closeness is measured, usually cosine similarity or dot product. The choice should match the embedding model.
Approximate nearest neighbour (ANN)
Index structures such as HNSW that find very close matches fast without comparing against every stored vector.
Top-k
How many results to retrieve. Too few misses context; too many adds noise and cost to the prompt.
Metadata filtering
Restricting search to, for example, one department's documents or the latest version, before or during the similarity search.
Hybrid search
Combining vector similarity with keyword search, which helps with exact terms such as product codes and names.
Vector database tools to try first
You can learn the concepts with free, local tools before using a managed service:
- FAISS – a fast similarity search library from Meta, good for learning and experiments.
- Chroma – a simple open-source vector store that runs locally and is popular in RAG tutorials.
- pgvector – adds vector search to PostgreSQL, useful when your data already lives in a relational database.
- Qdrant, Weaviate and Milvus – open-source vector databases that also offer hosted versions.
- Pinecone – a fully managed vector database service.
Quick answers about vector databases
Short answers to the questions AI learners search for most.
What is a vector database in simple words?
It is a database that stores data as lists of numbers representing meaning and finds the stored items most similar in meaning to a query, rather than matching exact words.
Is a vector database required for RAG?
Not strictly. A small project can keep vectors in memory or in a file. A vector database becomes important when you have many documents, need filtering, updates and fast responses.
What is the difference between a vector database and a normal database?
A normal database is designed for exact matches and structured queries. A vector database is designed for similarity search over high-dimensional vectors, using special indexes to stay fast.
Which vector database should a beginner learn?
Start with Chroma or FAISS locally to understand embeddings and retrieval, then try pgvector or a hosted database once you build a full application.
Does a vector database stop LLM hallucinations?
It reduces them by giving the model relevant source text, but it does not remove them. Good chunking, retrieval checks, prompts that require citations and evaluation are still needed.
Go further with RAG and LLM applications
See where RAG sits in the syllabus, or read the guides that build on it.
Build a tiny RAG app this week
Take ten pages of your own notes, chunk and embed them into Chroma, and ask questions about them. Then change the chunk size and top-k and watch how the answers change. That one experiment teaches more than any diagram.

