Festival Season OfferUp to 25% off on all our programmes — claim it before you enrol
← All Career InsightsAI & ML

What is a vector database and how does RAG use it?

A vector database stores embeddings, lists of numbers that capture the meaning of text, images or other data, and finds the items whose meaning is closest to a query. In retrieval-augmented generation (RAG), it is the memory: your documents are split into chunks, embedded and stored, and at question time the most relevant chunks are retrieved and given to the language model to answer from.

Why ordinary search is not enough for AI applications

A keyword search finds documents that contain the same words as the question. People rarely use the same words as the document, though. "How do I get my money back?" should match a page titled "Refund policy", even though no word is shared.

Embeddings solve this. An embedding model turns each piece of text into a vector, so texts with similar meaning end up close together. A vector database is built to store millions of these vectors and return the nearest ones in milliseconds.

How a vector database fits into a RAG pipeline

The same five steps sit behind most document chatbots and AI assistants.

  1. Chunk the documents

    Split long documents into passages of a few hundred words, often with a small overlap so no idea is cut in half.

  2. Create embeddings

    Send each chunk to an embedding model and get back a vector that represents its meaning.

  3. Store vectors with metadata

    Save each vector in the vector database with its text and details such as source, date and access rights.

  4. Retrieve on each question

    Embed the user's question the same way and ask the database for the top few most similar chunks, optionally filtered by metadata.

  5. Generate the answer

    Put the retrieved chunks into the prompt and ask the language model to answer only from them, ideally with citations.

Ideas you need to understand vector search

A few terms come up in every vector database guide and interview.

Embedding

A fixed-length list of numbers produced by a model to represent meaning. Similar meanings give nearby vectors.

Similarity metric

How closeness is measured, usually cosine similarity or dot product. The choice should match the embedding model.

Approximate nearest neighbour (ANN)

Index structures such as HNSW that find very close matches fast without comparing against every stored vector.

Top-k

How many results to retrieve. Too few misses context; too many adds noise and cost to the prompt.

Metadata filtering

Restricting search to, for example, one department's documents or the latest version, before or during the similarity search.

Hybrid search

Combining vector similarity with keyword search, which helps with exact terms such as product codes and names.

Vector database tools to try first

You can learn the concepts with free, local tools before using a managed service:

  • FAISS – a fast similarity search library from Meta, good for learning and experiments.
  • Chroma – a simple open-source vector store that runs locally and is popular in RAG tutorials.
  • pgvector – adds vector search to PostgreSQL, useful when your data already lives in a relational database.
  • Qdrant, Weaviate and Milvus – open-source vector databases that also offer hosted versions.
  • Pinecone – a fully managed vector database service.

Quick answers about vector databases

Short answers to the questions AI learners search for most.

What is a vector database in simple words?

It is a database that stores data as lists of numbers representing meaning and finds the stored items most similar in meaning to a query, rather than matching exact words.

Is a vector database required for RAG?

Not strictly. A small project can keep vectors in memory or in a file. A vector database becomes important when you have many documents, need filtering, updates and fast responses.

What is the difference between a vector database and a normal database?

A normal database is designed for exact matches and structured queries. A vector database is designed for similarity search over high-dimensional vectors, using special indexes to stay fast.

Which vector database should a beginner learn?

Start with Chroma or FAISS locally to understand embeddings and retrieval, then try pgvector or a hosted database once you build a full application.

Does a vector database stop LLM hallucinations?

It reduces them by giving the model relevant source text, but it does not remove them. Good chunking, retrieval checks, prompts that require citations and evaluation are still needed.

Go further with RAG and LLM applications

See where RAG sits in the syllabus, or read the guides that build on it.

See the AI & ML programmeSee the Generative AI & LLM Application Engineering moduleRead: RAG vs fine-tuningRead: what agentic AI isRead: prompt engineering with examplesBrowse all Career Insights

Build a tiny RAG app this week

Take ten pages of your own notes, chunk and embed them into Chroma, and ask questions about them. Then change the chunk size and top-k and watch how the answers change. That one experiment teaches more than any diagram.

Train for a AI & ML role

The same programme, duration and fees, with the learning path built around one job role.

AI EngineerMachine Learning EngineerGenerative AI EngineerMLOps EngineerAI Solutions EngineerBackend Developer

Build RAG applications with a mentor

Ask our team how the AI & ML programme in Madhapur, Hyderabad covers embeddings, vector databases and RAG projects.

Our admissions team will call you back within 90 minutes.
AddressHouse No. 3-535, 3rd Floor A Section, LR Towers, 100 Feet Road, Ayyappa Society, Madhapur, Hyderabad, Telangana 500081