top of page
Abstract Shapes

INSIDE

PUBLICATIONS

Vector Databases Explained: Embeddings for Search

Updated: 3 days ago

Vector Databases Explained: Embeddings for Search
Vector Databases Explained: Embeddings for Search
UIT emblem

UIT University 365 Institute of Technology

Series AI Engineering

Duration 25 minutes

IT Engineering, AI and Applied AI, Data Science, Software Development, Digital Transformation


UNOP isochrone

UNOP Sound (University 365 Neuroscience Oriented Pedagogy)

Take five minutes to prepare your brain. Play the isochronous tone track (40Hz gamma frequency) with your eyes closed. Gamma-frequency tones before a learning session raise attention and make the material easier to absorb.

[Audio player: UNOP Pre-Lecture Isochrone (40Hz, 5 minutes)]

In this Lecture




The Hook: Why Search Changed Forever


You search for 'how to handle database connection errors' and get exactly what you need in 0.2 seconds. Not because the exact words match, but because the search engine understands what you mean. That is vector search.


Traditional search counts keyword matches. Vector search measures semantic similarity. Instead of asking 'does this document contain these words?', it asks 'is this document about the same thing as the query?' The result: searches that find relevant content even when the words are completely different.


Vector databases store and search these semantic representations. They power RAG pipelines, recommendation systems, image search, and duplicate detection. In the next 20 minutes, you will understand how they work, which ones to choose, and how to build a search system that understands meaning, not just keywords.


Vector database storing embeddings and performing similarity search
Vector database storing embeddings and performing similarity search


Back to the TOC

What Are Embeddings?


Embeddings are vectors (lists of numbers) that represent the semantic meaning of text, images, or audio. Text with similar meaning gets vectors that are close together in high-dimensional space. The distance between vectors measures semantic similarity.


A sentence like 'the cat sat on the mat' and 'the feline rested on the rug' have different words but nearly identical meaning. Their embedding vectors are close neighbors. This is why vector search finds relevant content without exact keyword matches.


Embeddings are produced by embedding models: OpenAI text-embedding-3-large, Cohere embed-v3, or open-source models like BGE-large. These models are trained on massive text corpora to map similar meanings to nearby points in vector space.


How text becomes vectors in high-dimensional space
How text becomes vectors in high-dimensional space


Back to the TOC

How Vector Search Works


Vector search finds the nearest neighbors to a query vector. You embed the query, compare it against all stored vectors, and return the closest matches. The comparison uses a distance metric.


Three common distance metrics:


- Cosine similarity: Measures the angle between vectors. Values range from 0 (identical direction) to 90 degrees (unrelated). Robust to vector magnitude differences. Most common for text search.


- Dot product: Measures the projection of one vector onto another. Faster than cosine but sensitive to vector magnitude.


- Euclidean distance: Measures straight-line distance between vectors. Less common for normalized embeddings but useful for some applications.


For most text search applications, cosine similarity is the right choice.


How Vector Search Works
How Vector Search Works: pedagogical overview


Back to the TOC

Popular Vector Databases in 2026


The vector database market has matured. Here are the main options:


- Pinecone: Managed cloud service. Easiest to start with. Good for teams that do not want to manage infrastructure.


- Weaviate: Open-source with managed cloud option. Supports hybrid search (vector + keyword) natively.


- Qdrant: Open-source, Rust-based, fast. Good for self-hosted deployments.


- pgvector: PostgreSQL extension. If you already use Postgres, this adds vector search without a new system.


- Chroma: Lightweight, designed for AI application development. Good for prototyping.


The choice depends on your infrastructure, scale, and whether you need managed or self-hosted. For prototyping: Chroma. For production with existing Postgres: pgvector. For large-scale managed: Pinecone. For self-hosted production: Qdrant or Weaviate.


Popular Vector Databases in 2026
Popular Vector Databases in 2026: pedagogical overview


Back to the TOC

Indexing Algorithms: HNSW and IVF


Comparing a query vector against every stored vector is too slow at scale. Indexing algorithms organize vectors to make search faster, at the cost of approximate results.


HNSW (Hierarchical Navigable Small World) builds a multi-layer graph where each node connects to its nearest neighbors. Search traverses the graph from top to bottom, narrowing to the correct neighborhood. HNSW is the default in most vector databases because it offers the best speed-to-accuracy trade-off.


IVF (Inverted File Index) clusters vectors into buckets. Search only checks vectors in the most relevant buckets. Faster but less accurate than HNSW.


Both algorithms trade exact search for approximate nearest neighbor (ANN) search. The accuracy loss is typically under 5% while speed improves by 10 to 100x.


Indexing Algorithms: HNSW and IVF
Indexing Algorithms: HNSW and IVF: pedagogical overview


Back to the TOC

Chunking Strategies for RAG


Before embedding documents, you need to split them into chunks. Chunking determines what the vector database stores and what the search retrieves.


- Fixed-size chunking: Split every N tokens (typically 400-600). Simple but can cut mid-sentence.


- Semantic chunking: Split at natural boundaries (paragraphs, sections, headings). Preserves meaning.


- Sentence-level chunking: One sentence per chunk. Fine-grained but may lose context.


- Parent-document chunking: Embed small chunks for precise search, but retrieve the parent document for context.


Add 15% overlap between chunks to preserve context across boundaries. Without overlap, a concept split across two chunks is unsearchable.


Four chunking strategies compared with examples
Four chunking strategies compared with examples


Back to the TOC

Hybrid Search: Combining Vector and Keyword


Vector search finds semantic matches. Keyword search finds exact matches. Some queries need both.


A product code like 'SKU-48291' has no semantic neighbors. A vector search for it fails. But a keyword search finds it instantly. Hybrid search combines both: BM25 for keyword matching and vector similarity for semantic matching, then merges the results.


Most production search systems use hybrid search. Weaviate supports it natively. Pinecone and Qdrant added hybrid search in 2025-2026. If your vector database does not support hybrid natively, you can run BM25 separately and merge results in your application code.


How hybrid search combines vector and keyword search results
How hybrid search combines vector and keyword search results


Back to the TOC

Choosing an Embedding Model: The Decision That Fixes Your Ceiling


Every other decision in a vector search system is a tuning exercise. The embedding model is different: it sets the ceiling on what your search can find, and no index parameter can recover relevance the model never encoded.


The practical selection process has four steps, and the first one is not "check the leaderboard".


Step 1: Fix the input language mix


An English-only model on a corpus that is 30 percent French will silently fail on that 30 percent. Multilingual capability is not a nice-to-have; it is a correctness requirement that is invisible in an English-only evaluation set. Determine the language distribution of your corpus and your queries before you look at any benchmark.


Step 2: Fix the input length distribution


Every embedding model has a maximum input length, and text beyond it is truncated, usually without an error. A model with an 8,192-token limit and a corpus of 20-page policy documents will embed the first few pages and discard the rest. If your chunks are longer than the model's limit, you are not measuring the model you think you are measuring.


Measure the length distribution of your chunks, then choose a model whose limit sits above the 95th percentile, not the median.


Step 3: Compare on retrieval, not on a general score


Public leaderboards average performance across dozens of task types. Your task is one of them, and it is not the average. A model that leads on a general ranking can lose to a cheaper alternative on your specific data.


The correct comparison is a small evaluation on your own corpus:


  • Collect 50 to 200 real queries from your users or from your logs.

  • For each query, mark which documents are relevant. This is tedious. It is also the only thing that makes the comparison meaningful.

  • Embed the corpus with each candidate model and run the same queries.

  • Compare recall at the cut-off you actually serve, typically the top 5 or top 10 documents.


A two-hour evaluation on 100 real queries beats any published benchmark, because it measures the interaction between your data, your queries and the model, which no leaderboard can.


Step 4: Weigh cost, latency and dimension count


Three practical numbers decide the rest:


  • Dimensions. Every dimension is storage and distance computation. A 3,072-dimension model stores twice as much as a 1,536-dimension model and scores each comparison twice as expensively.

  • Cost per million tokens. Embedding is cheap compared to generation, but you pay it on your entire corpus plus every query, and you pay it again on every re-index.

  • Latency. Query embedding sits directly in the user's request path. A 200-millisecond embedding call is 200 milliseconds added to every search.


Matryoshka embeddings: one model, several lengths


Modern embedding models are trained with Matryoshka Representation Learning, which orders the information in a vector so that the first N dimensions carry most of the meaning. The practical consequence is that you can truncate the vector and keep most of the quality.


This lets you run one model at two settings: full length in the vector store for accuracy, and a truncated prefix for a fast first-stage search. It also lets you shrink storage without changing models, which is often easier to approve than a model migration.


The rule to remember is that truncation is supported by the model, not by the database. Truncating a vector from a model that was not trained for it produces meaningless results, and it produces them silently.


The evaluation trap


Do not evaluate embedding models with the queries you used to build the system. Those queries are biased toward whatever your first implementation already retrieves. Take the evaluation set from production logs, or from a person who has never seen the system, and freeze it before you compare anything.


The four-step embedding model selection process, from language mix and chunk length to a retrieval evaluation on 100 real queries
The four-step embedding model selection process, from language mix and chunk length to a retrieval evaluation on 100 real queries



Back to the TOC

Measuring Retrieval Quality: Recall, Precision and the Numbers That Matter


You cannot improve retrieval you do not measure, and the metric most teams reach for first is the wrong one.


The two metrics that matter


Recall at K answers: of the documents that are relevant to this query, how many appear in the top K results? If three documents are relevant and your top 10 contains two of them, recall at 10 is 0.67.


Precision at K answers: of the K documents you returned, how many are relevant? If your top 10 contains two relevant documents and eight irrelevant ones, precision at 10 is 0.20.


For a retrieval-augmented generation system, recall is the metric that decides answer quality. A missing relevant chunk cannot be compensated for by the language model, while an extra irrelevant chunk usually can. Optimise recall first and use precision to trim the context you send.


The metric that captures rank


Recall and precision both ignore where in the list a result landed. Mean Reciprocal Rank captures it: for each query, take the position of the first relevant result, invert it, and average across queries. A first relevant result at position 1 scores 1.0, at position 2 scores 0.5, at position 5 scores 0.2.


MRR is the right headline number for a search box where the user wants one good answer. Recall at K is the right headline number for a RAG pipeline where the reader wants every relevant passage.


A measurement harness you can build in an afternoon


queries 150 (drawn from real traffic) qrels query_id -> set of relevant document ids metric recall@10, precision@10, MRR@10 report one row per change: index=hnsw m=16 ef_search=40 recall@10=0.87 mrr@10=0.71 index=hnsw m=16 ef_search=120 recall@10=0.94 mrr@10=0.76


Keep every row. The history is more valuable than any single number, because it tells you which changes were worth the cost.


The ANN accuracy question


Approximate indexes return approximate answers. The loss is real and it is measurable: run the same queries against an exact search and against your approximate index, and report the difference.


A useful discipline is to keep an exact-search baseline for a small sample of your queries. It costs a sequential scan over the sample, which is fine for 100 queries, and it gives you an upper bound: if your approximate index recalls 0.94 and exact search recalls 0.96, then 0.94 is close to the achievable maximum and further index tuning has little headroom. Without that baseline you cannot tell whether an index change improved things or merely reshuffled the same misses.


Where the evaluation set goes wrong


The failure mode to watch for is a set that is too easy. If recall at 10 is already 0.99 before you have tuned anything, the set is not discriminating and every change will look neutral. The fix is to add the hard queries deliberately: the query phrased in a user's own words rather than the document's vocabulary, the query whose answer spans two documents, the query that expects a document with no exact keyword overlap.


Those queries are exactly the ones your system will fail on in production, so they belong in the set from the start.




Back to the TOC

Index Tuning: The Knobs That Actually Move Recall and Cost


An approximate nearest neighbour index has two build-time parameters and one query-time parameter. Understanding which one to turn saves you a rebuild.


The three knobs


Parameter

When it applies

What it controls

Cost of raising it

`m`

Build time

How many neighbours each node in the graph connects to

More memory per vector, slower build

`ef_construction`

Build time

How wide the candidate list is while the graph is built

Slower build, no query-time cost

`ef_search`

Query time

How wide the candidate list is while a query runs

Higher latency per query


The most important fact about this table is that ef_search is a runtime setting. You can change recall and latency per query without rebuilding the index, which means the accuracy-versus-speed decision is a live configuration choice rather than an architectural commitment.


The corollary matters just as much: m and ef_construction can only be changed by rebuilding. Get them right once, and use ef_search for everything after that.


A starting configuration


For a text corpus with embeddings in the 768 to 1,536 dimension range, a widely used starting point is:


m = 16 ef_construction = 128 to 200 ef_search = 40 initially, tuned upward until recall meets the target


Then measure. Raise ef_search in steps and record recall and p95 latency at each step. Stop where recall meets your target; the remaining headroom is bought with latency you do not need to spend.


If recall will not reach the target even at a high ef_search, the graph quality is the limit and you need a rebuild with a higher m or a higher ef_construction. That is the diagnostic value of tuning ef_search first: it tells you whether the problem is at query time or at build time.


Memory is the real constraint


An HNSW index stores the graph in memory. The working set is the vector data plus the graph edges, and if it does not fit in memory the queries fall back to reading from disk and the latency stops resembling your benchmark.


Two arithmetic facts to plan with:


  • A full-precision vector at 1,536 dimensions costs about 6 kilobytes per row of raw vector data, before the graph.

  • The graph multiplies the index size by roughly 1.5 at m=16, so the same row costs about 9 kilobytes in the index.


Ten million rows at that size is on the order of 90 gigabytes of index. That number decides your instance type, and it decides whether you can afford full precision at all.


Quantization: the highest-value storage move


Storing vectors at half precision instead of full precision roughly halves the storage and the memory pressure, with a recall change that is small on typical text embeddings.


full precision (float32) 4 bytes per dimension half precision (float16) 2 bytes per dimension binary (1 bit per dim) 0.125 bytes per dimension, needs a re-ranking stage


The progression to follow is incremental, not maximal: try half precision first and measure recall. If storage is still the constraint, move to a two-stage design where a heavily compressed index retrieves a large candidate set and a full-precision re-ranking pass orders the top of that set. Two-stage search is the standard way to get both a small index and accurate results, at the cost of one extra step in the query.


Do not start with the most aggressive compression. Start with the least lossy option that solves the memory problem.


The filtered-search trap


A query with a metadata filter is a different problem from a plain nearest-neighbour query. If the filter is selective, a graph traversal can walk into a region where almost nothing matches the filter, and it returns fewer results than you asked for while reporting success.


The symptom is a query for the top 10 that returns three. The cause is that the index found 10 nearest neighbours and then the filter discarded seven of them, with no further search.


Two mitigations, both standard:


  • Iterative scanning. When the filter is applied after the index walk, allow the search to continue past the first candidate set until enough matching results are collected or a scan budget is exhausted.

  • Pre-filtering. Where the filter is selective enough, use a filtered index or a partial index so the search never enters the excluded region.


Whichever you use, add a check to your harness: assert that a filtered query returns its requested count. A silent short result is one of the most common production defects in vector search, and it is invisible without an explicit assertion.




Back to the TOC

Chunking in Production: A Worked Design


Chunking is the step where a good embedding model and a good index meet a bad decision and produce a bad system. It is also the step most teams implement first and revisit never.


What chunking actually decides


A chunk is the unit of retrieval. Whatever you put in a chunk is what the system can find, and whatever you split is what the system cannot connect. Two chunks that each contain half of an answer will both rank mediocrely for a question the whole answer would have answered well.


The four strategies and when each is right


Fixed-size with overlap. Split every N tokens, with a percentage of overlap. Simple, predictable, and a reasonable default. Set N from your embedding model's limit and your average paragraph length, not from a round number, and use an overlap of about 10 to 15 percent so that a concept crossing a boundary appears whole in at least one chunk.


Structure-aware. Split on the document's own structure: headings, sections, list boundaries. This is the strongest option for documentation, policy and reference material, because the structure was put there by a human who was organising meaning. It requires your ingestion pipeline to parse the format rather than to treat it as flat text.


Semantic. Split where the topic changes, using embedding similarity between adjacent sentences as the signal. It produces the most coherent chunks and costs an embedding pass over every sentence, which is real money on a large corpus.


Parent and child. Index small chunks for precise matching, but return the larger parent passage or the whole section as context. This is the best of both worlds and it is what most mature systems converge on: precision at retrieval time, context at generation time.


The measurements that settle the argument


Chunking choices are testable. Build your evaluation set once, then run it against several chunk sizes and strategies and record the numbers.


strategy recall@10 avg chunk chunks/M docs fixed 400, no overlap 0.71 400 2,500 fixed 400, 15% overlap 0.83 400 2,875 structure-aware by section 0.91 620 1,600 parent-child (200 child, 900 p) 0.93 200 10,300 semantic 0.92 540 1,900


The numbers are illustrative; the pattern is what to look for. Overlap is nearly free recall for a modest increase in storage. Structure-aware chunking usually wins on documents that have structure. Parent-child usually wins on precision, and it costs the most index entries.


The metadata that makes chunks useful


A chunk is more useful when it carries context it cannot carry in its own text. Attach, at minimum:


  • the source document identifier and title

  • the section heading the chunk sits under

  • the position of the chunk in the document

  • the date of the source

  • any access-control attribute your system respects


The section heading matters more than it sounds. A chunk that reads "the timeout must not exceed 30 seconds" is ambiguous alone and unambiguous when accompanied by the heading "Connection pooling limits". Several teams improve retrieval measurably by prepending the heading to the chunk text before embedding it, at no extra model cost.


The two mistakes to avoid


Chunking by character count. Characters and tokens are different units, and a limit expressed in characters produces wildly variable chunk sizes across languages and formats.


Splitting the answer from the question. In a reference document, a question and its answer frequently sit in adjacent blocks. Splitting on a fixed size can put them in different chunks, and then neither chunk retrieves well for the question. Structure-aware splitting, or a rule that keeps a heading with its first body block, prevents it.




Back to the TOC

Reranking: The Second Stage That Usually Wins


A first-stage vector search is optimised for speed across millions of candidates. It is not optimised for the precision of the top five results. Reranking is the step that fixes the difference, and on most systems it is the single largest quality improvement available after the index is built.


How reranking works


  • The vector index retrieves a wide candidate set, typically 50 to 200 documents.

  • A reranker scores each candidate against the query with a much more expensive and much more accurate model.

  • The top few results of the reranked list become the context, or the answer to the user.


The economics are straightforward: a first-stage search over the whole corpus is fast but coarse, and a reranker over 100 candidates is slow but fine-grained. Splitting the two lets each stage do what it is good at.


The two kinds of reranker


Cross-encoders. A model reads the query and the document together and outputs a relevance score. This is the most accurate approach and the slowest, because the model runs once per candidate rather than once per query.


Late-interaction models. A model produces a token-level representation of the query and of each document, and the score is computed from the interaction between them. This can be precomputed for documents, so query-time cost is much lower than a cross-encoder while keeping much of the accuracy.


The trade-off to plan for is candidates times per-candidate latency. Reranking 200 candidates at 20 milliseconds each is four seconds, which is not a search experience. Reranking 50 at 20 milliseconds is one second, which is acceptable for a complex query and too slow for a keystroke-by-keystroke suggestion box. Measure with your own candidate counts.


When reranking is worth it


Reranking helps most when the failure you see is a good document at position 14. It helps least when the failure is that the document is not in the candidate set at all, because a reranker can only reorder what the first stage found.


That distinction is the diagnostic to run first. Take failing queries from your logs and check whether the correct document was anywhere in the first 200 results. If it was, reranking will convert the failure into a success. If it was not, the problem is upstream, in the embedding model, the chunking or the query formulation, and reranking will add latency for no gain.


Query transformation, the other upstream lever


Two low-cost transformations often raise recall before anything else is changed:


Hyde-style expansion. Generate a hypothetical answer to the query with a language model, then embed that answer text instead of the raw query. Answers and documents share vocabulary in a way that questions and documents often do not, so the embedding lands closer to the passages you want.


Multi-query expansion. Generate several paraphrases of the query, retrieve for each, and merge the result sets. It multiplies query cost by the number of paraphrases, and it reliably raises recall on short or ambiguous queries.


Both belong in the harness: build them, measure them on your evaluation set, and keep only the ones that move recall enough to justify their latency.




Back to the TOC

Operating a Vector Database: Cost, Growth and Migration


A vector index is not a static artefact. It grows with your corpus, it must be rebuilt when the embedding model changes, and it costs real money every month. The operational side is where most vector systems quietly become expensive.


The three recurring costs


Storage. Vector data plus the index graph plus the original documents and their metadata. A useful rule is that only about a third of your vector-store footprint is vectors; the rest is the text you are keeping for retrieval and the index structure itself.


Memory. The graph must be resident to deliver the latency you measured. If the index outgrows memory, latency degrades sharply and it degrades under load, so it appears as an intermittent problem rather than a clean failure.


Re-embedding. Every model change, every chunking change and every major corpus change means embedding the corpus again. At 10 million chunks of 400 tokens, that is 4 billion tokens of embedding work, and it is the largest single line item in most vector search budgets.


The migration question


Changing the embedding model is not a configuration change. The vectors from two different models are not comparable, so the old index cannot be reused and the corpus must be embedded again.


The safe migration pattern:


  • Build the new index alongside the old one, in a separate collection or namespace.

  • Run the retrieval evaluation against both.

  • Serve from the old index while the new one is being populated, so no query fails during the migration.

  • Switch reads over, watch the retrieval metrics on live traffic, and keep the old index for a defined period so the switch can be reversed.

  • Only then delete the old index.


The cost of this pattern is double storage during the migration. The benefit is that a failed migration is a rollback rather than an outage, and the benefit is worth the cost every time.


Re-indexing incrementally


A full rebuild of a large index can take hours, and during a rebuild the index is either unavailable or serving stale data. Plan for incremental maintenance:


  • Insert new documents as they arrive.

  • Delete documents when their source is removed, and verify that deletes actually reclaim space in your engine.

  • Update a document by delete plus insert, because most graph indexes do not support in-place vector updates efficiently.


The maintenance workload is easy to forget, and a corpus that only grows produces an index you cannot rebuild inside a maintenance window. Test your rebuild time before you need it.


A monitoring set worth having


index size on disk index resident in memory p95 query latency at the production ef_search filtered queries that returned fewer than the requested count recall@10 against the frozen evaluation set embedding cost per day stale documents: rows whose source was deleted upstream


Six numbers, checked on a schedule. Each one corresponds to a failure that will otherwise be discovered by a user: the first two by a latency complaint, the third by a slow search, the fourth by a wrong answer, the fifth by a budget overrun, and the sixth by a result the user should no longer be able to see.




Back to the TOC

A Worked Example: Building Internal Documentation Search


The following walks a realistic project end to end. The corpus is 12,000 pages of internal engineering documentation: runbooks, architecture notes, onboarding material and incident reports.


Day one: the evaluation set


Before building anything, collect 150 real questions from the team's support channel. For each, record the page or pages that answer it. This takes a few hours and it feels like a delay. It is the only step that makes every later decision measurable.


Add the hard cases deliberately:


  • questions phrased in a newcomer's vocabulary, where the document uses internal jargon

  • questions whose answer is spread across a runbook and an architecture note

  • questions that mention a system by an old name that was renamed

  • questions about a procedure that was deleted on purpose, where the correct answer is "this no longer applies"


That last category is important. A retrieval system that confidently returns the obsolete runbook is worse than one that returns nothing.


Day one: the first pipeline


parse structure-aware: split on headings, keep the heading with its section chunk 400 to 900 tokens depending on section length, 12% overlap embed 768-dimension model, chosen after a comparison on 100 of the 150 questions index HNSW, m=16, ef_construction=160, ef_search=40 store the chunk plus the source path, section heading, last-modified date


Record the baseline numbers immediately: recall at 10 and MRR at 10. Without a baseline you will not know whether any later change helped.


Day two: the first round of tuning


Run each change separately and record the score:


  • Prepend the section heading to the chunk text before embedding. Usually the cheapest real gain available.

  • Raise ef_search from 40 to 100. Record the recall gain and the latency cost.

  • Add a reranker over the top 50 candidates. Measure the improvement against the added latency.

  • Add a metadata filter on last_modified for queries that mention a specific quarter, and assert that filtered queries still return their requested count.


Stop when the next change costs more than the improvement is worth. That point is different for every system, which is why the harness exists.


Day three: the operating plan


  • Re-embed the corpus when a document changes, not on a fixed schedule.

  • Rebuild the index when the corpus grows by more than a set percentage, and test the rebuild time.

  • Re-run the evaluation set monthly, and on every model or chunking change.

  • Alert when a filtered query returns fewer results than requested.

  • Keep the previous index for one week after a model migration.


What the project actually teaches


Three things, in order of importance:


  • The evaluation set came first, and it is the reason every later decision was a measurement rather than an opinion.

  • The largest single gain came from the smallest change, prepending the section heading, because it fixed an information problem rather than a model problem.

  • The obsolete-document case was the one that needed an explicit product decision, since the system can only know a procedure was withdrawn if the ingestion pipeline records that.


Reader exercise


Take one collection of documents you own. Write ten queries you would genuinely ask of it, mark the correct answers by hand, and measure recall at 5 on whatever search you have today. Ten queries is enough to find out whether the system is at 0.9 or at 0.4, and those are two very different projects.




Back to the TOC

Query Understanding: The Half of Search Nobody Tunes


Most retrieval work is spent on the index and the embedding model. Both of those act on whatever text arrives. If the query arriving is ambiguous, the best index in the world returns the best answer to the wrong question.


Why queries fail before they reach the index


Production queries differ from the queries a developer types during testing. They are short, they contain typos, they use the user's own vocabulary rather than the document's, and they assume context the search engine does not have.


Four query shapes account for most retrieval failures:


The vocabulary mismatch query. The user asks about a "login loop", and the document is titled "session re-authentication handling". The concepts are identical and the words share nothing. This is the case embeddings help with most, and it is also the case they only partly solve, because a term that appears nowhere in the corpus has no strong semantic neighbour.


The underspecified query. The user asks "how do I fix the timeout". There are forty timeouts in the system. The engine answers with whichever section embeds closest to the question, which is often arbitrary. The fix is conversational: ask which service.


The multi-part query. The user asks "what changed in the retry policy and who owns the billing service". One query, two intents. A single vector cannot represent both well, because the embedding of the combined text sits between the two topics instead of on either. Splitting the query and merging the results usually beats trying to embed it whole.


The negative query. The user asks "why is this NOT working". Negation is weakly represented in most embedding spaces, because a passage about a failure and a passage about a fix use almost the same words. Explicit handling beats hoping the embedding captures the polarity.


The transformations worth building


Each of these is a small amount of code, and each is testable on your evaluation set.


Query rewriting before embedding. A short language-model call rewrites the user's query into the vocabulary of the corpus and resolves pronouns from the conversation history. For a conversational search interface, this is often worth more than any index change.


HyDE-style hypothetical answers. Generate a plausible answer to the query, then embed the answer rather than the question. Answers and documents share structure, so the embedding lands nearer the passages you want. The generated answer does not need to be correct, because it is never shown to the user; it only needs to be on-topic.


Multi-query expansion. Generate several paraphrases, retrieve for each, and merge. This raises recall most on short queries, and it multiplies query cost by the number of paraphrases, so three is usually the right number.


Query classification. A cheap classifier routes the query: a product code goes to keyword search, a natural-language question goes to vector search, a conversational follow-up goes through the rewriter first. Routing is boring and it removes a whole class of failures.


The measurement that keeps you honest


Every transformation above adds latency. The only reason to keep one is that it moves recall on your frozen evaluation set by more than the latency it costs. Record both numbers on the same row:


change recall@10 p50 query latency baseline 0.71 120 ms + query rewrite 0.83 310 ms + multi-query expansion (3 paraphrases) 0.88 640 ms + reranker over top 50 0.93 910 ms


Read the table as a budget, not as a leaderboard. Three hundred milliseconds of latency for twelve points of recall on a documentation search is a good trade. The same latency for four points on a suggestion box that fires on every keystroke is not.





Back to the TOC

Feynman Summary: Explain It Like You Are 12


Imagine every document in your library is converted into a point on a giant map. Documents about the same topic end up close together on the map. Documents about different topics end up far apart.


When you search for something, your question also becomes a point on the map. The search finds whatever documents are closest to your question's point. This works even if you use completely different words, because the map is based on meaning, not spelling.


A vector database is the map. Embeddings are the coordinates that place each document on the map. Similarity search is finding the nearest points. Chunking is deciding how big each piece of the document should be before placing it on the map.


Hybrid search is using both the meaning map and a regular keyword index. The keyword index catches exact matches (product codes, names, error messages). The meaning map catches conceptual matches. Together they find everything.



Back to the TOC

Mindmap: The Complete Picture


Complete mindmap of Vector Databases Explained: Embeddings for Search
Complete mindmap of Vector Databases Explained: Embeddings for Search

This mindmap shows the key concepts, relationships, and decision points covered in this lecture.



Back to the TOC

UNOP isochrone

UNOP Sound (University 365 Neuroscience Oriented Pedagogy)

Take five minutes to consolidate your memory. Play the isochronous tone track (10Hz alpha frequency) with your eyes closed. Alpha-frequency tones after a learning session support consolidation, helping move what you just learned from short-term to long-term memory.

[Audio player: UNOP Post-Lecture Isochrone (10Hz, 5 minutes)]


Practical Exercise: Compare Vector Databases


Research and compare three vector databases for a hypothetical project.


Step 1: Define Your Use Case


Write one sentence: what will you search, how many documents, and what is your budget?


Step 2: Compare These Options


- Pinecone (managed, cloud)


- Qdrant (self-hosted, open-source)


- pgvector (PostgreSQL extension)


Step 3: Evaluate


For each option, write:


1. Setup time (hours/days)


2. Cost per month at your scale


3. Does it support hybrid search?


4. Does it support filtering (metadata)?


5. What is the maximum vector dimension?


Step 4: Choose


Pick one and write one sentence explaining why. This exercise gives you a practical decision framework for vector database selection.



Back to the TOC

Glossary


Term

Definition

Embedding

Vector representation of text, images, or audio that captures semantic meaning in high-dimensional space.

Vector Database

Specialized database that stores and searches high-dimensional vectors using similarity metrics. Examples: Pinecone, Weaviate, Qdrant, pgvector.

Cosine Similarity

Distance metric measuring the angle between two vectors. The most common metric for text similarity.

HNSW

Hierarchical Navigable Small World: graph-based indexing algorithm that enables fast approximate nearest neighbor search.

IVF

Inverted File Index: clustering-based indexing algorithm that groups vectors into buckets for faster search.

ANN (Approximate Nearest Neighbor)

Search algorithm that finds approximately the closest vectors, trading accuracy for speed.

Chunking

Process of splitting documents into smaller passages before embedding. Semantic chunking preserves meaning better than fixed-size.

Hybrid Search

Search approach combining vector similarity (semantic) and BM25 (keyword) to catch both types of matches.

BM25

Sparse keyword matching algorithm that scores documents by term frequency and inverse document frequency.

Re-Ranker

Cross-encoder model that re-orders search results by true relevance to the query.

Embedding Model

Model that converts text into vector representations. Examples: OpenAI text-embedding-3, Cohere embed-v3, BGE-large.

Dimension

Number of values in an embedding vector. Typical: 768 to 3072 dimensions. Higher dimensions capture more nuance but cost more.

RAG

Retrieval Augmented Generation: technique that retrieves relevant documents from a vector database and passes them to an LLM.

Pinecone

Managed cloud vector database. Easiest to start with, no infrastructure management.

Weaviate

Open-source vector database with native hybrid search support.

Qdrant

Open-source, Rust-based vector database optimized for speed.

pgvector

PostgreSQL extension that adds vector similarity search to existing Postgres databases.

Chroma

Lightweight vector database designed for AI application prototyping.

UNOP

University 365 Neuroscience-Oriented Pedagogy: the teaching framework behind this lecture format.



Back to the TOC

Quiz: TEST YOUR UNDERSTANDING


1. What is the main advantage of vector search over keyword search?


A) Vector search is faster


B) Vector search finds semantically similar content even with different words


C) Vector search does not require an index


D) Vector search works without embeddings


2. Which distance metric is most commonly used for text similarity?


A) Euclidean distance


B) Manhattan distance


C) Cosine similarity


D) Hamming distance


3. What does HNSW stand for and what is it used for?


A) High-speed Network Search Web: for web search


B) Hierarchical Navigable Small World: for fast approximate nearest neighbor search


C) Hybrid Node Search Worker: for distributed search


D) Hash-based Natural Search Word: for keyword search


4. Why should you add overlap between chunks when chunking documents?


A) To increase the number of chunks


B) To preserve context across chunk boundaries


C) To reduce embedding costs


D) To improve keyword matching


5. When should you use hybrid search instead of pure vector search?


A) Always: hybrid is always better


B) Never: vector search is sufficient


C) When you need to match exact keywords (product codes, names) alongside semantic matches


D) Only when your vector database does not support HNSW


Answers: 1-B, 2-C, 3-B, 4-B, 5-C



Back to the TOC

Related Resources


U365 INSIDE Publications


- How LLMs Actually Work: Transformers in 20 Minutes (AI Foundations, Lecture 1)


- RAG vs Fine-Tuning: When to Use Each (AI Engineering, Lecture 2)


- Building Your First AI Agent with Function Calling (AI Agents, Lecture 3)


External Resources


- Research papers and official documentation for topics covered in this lecture


- Open-source tools and libraries referenced in the content


Related U365 Lectures (Coming Soon)


- Additional lectures in the AI Engineering series


- Cross-referenced lectures from AI Engineering and AI Foundations series



Back to the TOC

U.Copilot for This Lecture


Copy and paste this prompt into the U.Copilot AI Agent on university-365.com to explore this topic further:


I just completed the U365 INSIDE Lecture "Vector Databases Explained" from UIT. Help me: 1. Choose a vector database for my use case 2. Design a chunking strategy for my documents 3. Decide if I need hybrid search 4. Estimate the cost and infrastructure I will need 5. Suggest an embedding model for my language and content type My use case is: [describe your project here]



Back to the TOC

Next Steps


1. Take the quiz above and check your answers at the bottom of this section.


2. Complete the Practical Exercise: compare three vector databases for your use case.


3. Read Lecture 2 (RAG vs Fine-Tuning) to understand how vector databases fit into RAG pipelines.


4. Read Lecture 5 (Prompt Engineering at Production Scale) to learn how to use retrieved context effectively.


5. Visit university-365.com/uit to explore UIT programs in Data Science and AI Engineering.


Answers: 1-B, 2-C, 3-B, 4-B, 5-C



Back to the TOC

IMPORTANT NOTICE


Copyright University 365, Inc. All rights reserved.


This lecture is part of the U365 INSIDE Lectures series, produced by UIT (University 365 Institute of Technology) under the UDA Department of Academics. The content follows the UNOP (University 365 Neuroscience-Oriented Pedagogy) framework and the 5M2S (5 Minutes to Success) microlearning format.


All lectures in this series are free to access. For enrollment in UIT degree programs, certificate programs, or executive education, visit university-365.com/tuition.


For permissions or inquiries, contact uda@university-365.com.


This content is for educational purposes. Technical details reflect publicly available information as of September 2026 and may change. Always consult official documentation before making architecture decisions.



Back to the TOC

Published by the Department of Academics, University 365.

Lecture delivered by the University 365 Institute of Technology (UIT).

Sam Utteker, Dean of Technology, UIT

Signed for the academic year 2026.

Comments

Rated 0 out of 5 stars.
No ratings yet

Add a rating
Image by Erik  Lucatero

Become Superhuman

Master AI to stay irreplaceable in every field.

 

 

 

​

​

Apply for Admission Today.
Select Your Initial Access Level.


Become a DISCOVERY, INSIDER, or SUPERHUMAN Fellow.

Image by Milad Fakurian

Master Your Life with a Digital Second Brain

Turn overwhelm into clarity with LIPS + CARE
U365’s unique framework to organize your goals, projects, and knowledge into a superhuman system for success

bottom of page