Vector DB

In-memory arrays teach concepts; a DB teaches product constraints

Your numpy index from lesson 5.1 is perfect for learning cosine similarity. Production RAG adds persistence, concurrent queries, metadata filters, and incremental updates — concerns a vector database (or vector-capable search engine) handles.

This lesson moves from list[np.ndarray] to a real service via Docker Compose, so you experience ops realities: volumes, healthchecks, connection strings, and index rebuild times.

Popular options for the course:

• Qdrant — straightforward REST/gRPC, good Docker story, rich filtering.
• Weaviate — schema-first, hybrid search built-in.
• pgvector — Postgres extension when you already run Postgres.

Pick one; concepts transfer. Examples below use Qdrant-style APIs generically.

Callout — the DB stores vectors + metadata, not trust: Garbage chunks with perfect indexing still produce wrong answers. Eval retrieval separately from generation.

What a vector DB provides

ANN algorithms (HNSW, IVF) trade exact recall for speed — tune ef_construct / m parameters in docs when datasets grow.

Docker Compose skeleton

# m5/compose/docker-compose.yml
services:
  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "6333:6333"
    volumes:
      - qdrant_data:/qdrant/storage
volumes:
  qdrant_data:

docker compose up -d then verify curl localhost:6333/healthz (exact path varies by version — check docs).

Never commit API keys into compose files; embedding calls stay in your ingest script using host .env.

Collection schema design

Define upfront:

• Vector size — must match embedding model output dimension.
• Distance metric — cosine vs dot vs euclidean; match normalization strategy.
• Payload fields — text, source_id, chunk_index, title, tags, ACL fields if needed.

Example create collection (conceptual):

client.create_collection(
    collection_name="docs",
    vectors_config=VectorParams(size=384, distance=Distance.COSINE),
)

Ingest loop:

for chunk in chunks:
    vector = embed(chunk.text)
    client.upsert(
        collection_name="docs",
        points=[PointStruct(
            id=chunk.id,
            vector=vector,
            payload={"text": chunk.text, "source_id": chunk.source_id, ...},
        )],
    )

Batch upserts (100–500 points) for throughput.

Querying with filters

Product queries rarely pure k-NN. Example: search refund policy only in lang=en:

client.search(
    collection_name="docs",
    query_vector=embed(user_query),
    query_filter=Filter(must=[
        FieldCondition(key="lang", match=MatchValue(value="en")),
        FieldCondition(key="category", match=MatchValue(value="billing")),
    ]),
    limit=5,
)

Filters applied before or during graph traversal depending on engine — read your DB docs to avoid "filter then brute force" latency surprises.

Ops concerns

Persistence: Named volumes survive container recreate. Back up volume snapshots before destructive experiments.

Reindexing: Changing embedding model requires full re-embed — version your index: docs_v2_384_miniLM.

Memory: HNSW indexes RAM-scale with vector count; monitor container memory on large corpora.

Idempotency: Upsert by stable chunk id so re-ingest does not duplicate.

Secrets: Compose on laptop is fine; production adds TLS, auth, network policies.

Callout — healthchecks in CI: A smoke test that upserts one point and queries it catches broken compose configs early.

Incremental updates and deletes

Real corpora change. Patterns:

Upsert on chunk id: Content hash in metadata — if hash unchanged, skip re-embed.

** Tombstone deletes:** Mark deleted=true in payload; filter at query time until compaction job removes vectors.

Full rebuild: Schedule weekly for small corpora; acceptable when <10k chunks and embed API is cheap.

Dual-write during migration: When switching embedding models, run parallel collections (docs_v1, docs_v2), query both behind feature flag, compare recall before cutover.

Document your update SLA in OPS.md: "new docs searchable within 5 minutes" implies async ingest worker, not synchronous upload blocking HTTP.

Query patterns and filters

Product queries combine vector search with business logic:

client.search(
    collection_name="docs",
    query_vector=vec,
    query_filter=Filter(must=[
        FieldCondition(key="team_id", match=MatchValue(value=user.team_id)),
        FieldCondition(key="status", match=MatchValue(value="published")),
    ]),
    limit=10,
)

Multi-tenant RAG must filter by tenant id — unfiltered k-NN leaks other customers' chunks into prompts. Test cross-tenant isolation explicitly in milestone eval.

Backup and disaster recovery

Snapshot vector volume before bulk reingest:

docker run --rm -v qdrant_data:/data -v $(pwd)/backups:/backup alpine tar czf /backup/qdrant_$(date +%F).tgz /data

Document restore procedure in OPS.md — reviewers appreciate ops realism even on toy compose stacks.

Local dev vs staging

Use Docker Compose locally with ephemeral volume for experiments; staging uses named volume + backup script. Never point dev ingest scripts at production DB URLs — classic footgun when .env.staging leaks into shell session.

Add Makefile targets: make db-up, make ingest, make query Q="refund policy" so reviewers reproduce without reading Python argparse.

Migration from in-memory index

Refactor shared interface:

class VectorStore(Protocol):
    def upsert(self, points: list[Point]) -> None: ...
    def search(self, vector: np.ndarray, k: int, filters: dict) -> list[Hit]: ...

Implement MemoryStore and QdrantStore. Swap in tests vs prod — same ingest pipeline.

Engineering problem (staff framing)

ANN indexes trade recall for latency/memory. Ops matter as much as HNSW params.

Diagram — Vector DB path

flowchart LR
  Chunks --> Emb --> Index[HNSW/IVF]
  Query --> EmbQ --> Index --> Hits

Precise definitions & mental model

ANN, HNSW/IVF/PQ, metadata filters, hybrid with keyword.

Tradeoffs — when to use what

In-memory (fast, costly) vs disk-backed; managed vs self-host.

Failure modes (interview + on-call)

Filter after ANN incorrectly; dim mismatch; no backup/rebuild plan.

Production & OSS practices

SLOs on p95 latency; rebuild runbooks; tenancy isolation.

Micro-project: Compose + ingest

In m5/vector_db/:

1. Add docker-compose.yml (or reference shared compose) for your chosen vector DB.
2. ingest.py — read chunked docs from lesson 5.2, embed, upsert with metadata.
3. query.py — CLI query returning top-5 payloads + scores.
4. OPS.md — notes on persistence volume, reindex procedure, filter example you used.

Document startup: docker compose up -d && uv run python ingest.py.

Checklist

• DB survives docker compose down / up with volume
• Ingest idempotent on stable ids
• Query returns text payload, not just ids
• OPS.md lists one operational gotcha you hit

Browse track companion

For the conceptual atlas (what a vector DB is, how ANN helps LLMs, Pinecone/Weaviate/Chroma/pgvector choice), read:

• /learn/data-databases-for-ai/vector-databases
• /learn/data-databases-for-ai/choosing-vector-stores

This lesson remains the hands-on Compose + ingest lab.