Chapter 45: NoSQL & Modern Data Stores

Why Theory Matters Before Picking a Tool

Every distributed data system makes explicit or implicit trade-offs. Without a vocabulary for those trade-offs, technology choices become folklore. Two complementary frameworks — the CAP theorem and PACELC — give practitioners a principled lens.

The CAP Theorem

Eric Brewer's CAP theorem (formalized by Gilbert and Lynch, 2002) states that a distributed data store can guarantee at most two of the following three properties simultaneously:

• Consistency (C): Every read receives the most recent write or an error.
• Availability (A): Every non-failing node returns a response (not an error), though it may be stale.
• Partition Tolerance (P): The system continues operating when network partitions (message loss between nodes) occur.

Because network partitions are a physical reality in any distributed deployment, P is non-negotiable in practice. The real choice is therefore CP vs. AP: do you prefer to reject a request rather than return stale data (CP), or do you prefer to serve a potentially stale response rather than return an error (AP)?

The PACELC Extension

CAP only describes behavior during a partition. Daniel Abadi's PACELC (2012) fills the gap by also characterizing the latency vs. consistency trade-off under normal (non-partition) operation:

• If there is a Partition: choose between Availability and Consistency (the CAP choice).
• Else (normal operation): choose between Latency and Consistency.

A system like Apache Cassandra is PA/EL — it favors availability during partitions and low latency in normal operation (at the cost of eventual consistency). HBase is PC/EC — it prioritizes consistency in both scenarios at the cost of availability and latency.

A Map of the NoSQL Families

The five major families differ in data model, query expressiveness, and CAP placement:

• Document stores (MongoDB, Couchbase): store JSON-like documents, rich query languages, flexible schema. Typically CP (MongoDB with majority write concern) or tunable.
• Key-value stores (Redis, DynamoDB): map opaque keys to values; extreme read/write throughput; limited query expressiveness. Redis is single-node CP; DynamoDB is tunable AP.
• Wide-column stores (Apache Cassandra, HBase): two-level key structure (partition key + clustering columns); designed for write-heavy time-series or event streams. Cassandra is AP/EL by default with tunable consistency.
• Graph databases (Neo4j, Amazon Neptune): native storage for nodes and edges; Cypher or Gremlin traversal languages; typically CP, single-node or causal cluster.
• Vector databases (Chroma, Pinecone, Weaviate, pgvector): store high-dimensional embedding vectors; primary query is approximate nearest-neighbor (ANN) search. Consistency models vary by engine.

Relational vs. NoSQL: A Decision Framework

No store is universally superior. The right choice depends on answering these questions:

1. Data shape: Is the data naturally tabular with fixed schema, or does it have variable structure, nesting, or graph topology?
2. Query patterns: Do you need ad-hoc multi-table joins, or do you have a small number of known, high-frequency access patterns?
3. Scale axis: Do you need to scale writes horizontally across nodes (NoSQL strength), or do reads dominate (both handle this)?
4. Consistency requirements: Can the application tolerate eventual consistency and stale reads?
5. Operational budget: Distributed NoSQL clusters introduce significant operational complexity.

Polyglot persistence — using multiple stores together, each optimized for a specific use case — is common in modern architectures. A typical e-commerce system might use PostgreSQL for orders/inventory (ACID required), Redis for session caches and rate limiting, MongoDB for product catalogs with variable attributes, and a vector database for semantic product search.

# Illustration: connecting to three stores in one application
import redis
import pymongo
from cassandra.cluster import Cluster

# Redis for sub-millisecond cache
cache = redis.Redis(host='localhost', port=6379, decode_responses=True)

# MongoDB for flexible document storage
mongo_client = pymongo.MongoClient('mongodb://localhost:27017/')
db = mongo_client['ecommerce']

# Cassandra for high-throughput event ingestion
cassandra_cluster = Cluster(['127.0.0.1'])
session = cassandra_cluster.connect('analytics')

print("Polyglot persistence: all three clients connected")

Common Pitfalls

• Choosing NoSQL to avoid schema design. "Schemaless" does not mean "design-free" — it means the database does not enforce the schema, so your application must. Poor document design in MongoDB is harder to migrate than a bad relational schema.
• Ignoring consistency implications. Accepting eventual consistency is a product decision, not just a technical one. An AP system may expose users to stale inventory counts or double-bookings.
• Underestimating operational complexity. A Cassandra or Kafka cluster requires deep expertise to tune, monitor, and repair. Managed cloud services (Atlas, Astra, ElastiCache) significantly reduce this burden.

The Document Model

MongoDB stores data as BSON documents (Binary JSON), which map naturally to the objects used in application code. Unlike relational rows, documents can contain nested sub-documents and arrays — no joins required to retrieve a logically related aggregate.

The hierarchy is: Cluster → Database → Collection → Document. A collection is analogous to a table, but documents in the same collection need not share the same fields.

# A product document — variable attributes per product category
{
    "_id": ObjectId("64f2a1b3c5d7e8f901234567"),
    "sku": "LAPTOP-X1",
    "name": "ProBook 15",
    "price_usd": 1299.99,
    "specs": {
        "cpu": "Intel i7-1355U",
        "ram_gb": 16,
        "storage_gb": 512
    },
    "tags": ["laptop", "business", "ultrabook"],
    "in_stock": True
}

Core CRUD with PyMongo

import pymongo
from bson import ObjectId
from datetime import datetime

client = pymongo.MongoClient('mongodb://localhost:27017/')
db = client['shop']
products = db['products']

# INSERT
result = products.insert_many([
    {"sku": "LAPTOP-X1", "price_usd": 1299.99, "category": "laptop",
     "specs": {"ram_gb": 16}, "tags": ["business"], "in_stock": True},
    {"sku": "MOUSE-M3",  "price_usd": 49.99,   "category": "peripheral",
     "specs": {"wireless": True}, "tags": ["wireless"], "in_stock": True},
    {"sku": "DESK-D9",   "price_usd": 599.00,  "category": "furniture",
     "specs": {"width_cm": 160}, "tags": ["standing"], "in_stock": False},
])
print(f"Inserted {len(result.inserted_ids)} documents")

# READ — query operators mirror MongoDB's JSON query language
expensive_in_stock = list(products.find(
    {"price_usd": {"$gt": 100}, "in_stock": True},
    projection={"sku": 1, "price_usd": 1, "_id": 0}
))
print(expensive_in_stock)
# [{'sku': 'LAPTOP-X1', 'price_usd': 1299.99}]

# UPDATE — $set modifies fields without replacing the document
products.update_one(
    {"sku": "DESK-D9"},
    {"$set": {"in_stock": True, "restock_date": datetime(2024, 9, 1)}}
)

# DELETE
products.delete_one({"sku": "MOUSE-M3"})

Indexing Strategy

Every query that is not served by an index triggers a collection scan — reading every document. For large collections this is disqualifying. MongoDB supports single-field, compound, multikey (array), text, geospatial, and TTL (time-to-live) indexes.

# Compound index: queries filtering on category then sorting by price
products.create_index([("category", pymongo.ASCENDING),
                        ("price_usd", pymongo.DESCENDING)])

# Text index for full-text search across multiple fields
products.create_index([("name", pymongo.TEXT), ("tags", pymongo.TEXT)])
list(products.find({"$text": {"$search": "standing desk"}}))

# TTL index: auto-delete session documents after 1 hour
sessions = db['sessions']
sessions.create_index("created_at", expireAfterSeconds=3600)

Use explain() to verify index usage:

plan = products.find({"category": "laptop"}).explain()
print(plan['queryPlanner']['winningPlan']['inputStage']['indexName'])
# 'category_1_price_usd_-1'

Aggregation Pipelines

MongoDB's aggregation pipeline is a staged data transformation engine — conceptually similar to SQL GROUP BY with window functions, but expressed as a sequence of pipeline stages.

pipeline = [
    {"$match": {"in_stock": True}},             # Stage 1: filter
    {"$group": {                                 # Stage 2: aggregate
        "_id": "$category",
        "avg_price": {"$avg": "$price_usd"},
        "count":     {"$sum": 1}
    }},
    {"$sort": {"avg_price": -1}},               # Stage 3: sort
    {"$project": {                               # Stage 4: reshape output
        "category": "$_id",
        "avg_price": {"$round": ["$avg_price", 2]},
        "count": 1,
        "_id": 0
    }}
]

for doc in products.aggregate(pipeline):
    print(doc)

Advanced Stages

• $lookup: left outer join to another collection (use sparingly — if you need many lookups, consider whether a relational model fits better).
• $unwind: deconstructs an array field into individual documents, one per element.
• $facet: runs multiple sub-pipelines in parallel for faceted search results.
• $graphLookup: recursive graph traversal within a collection.

Schema Design Patterns

Embed vs. Reference is the central MongoDB design decision:

• Embed sub-documents when the data is accessed together, has a bounded size, and has a 1-to-few relationship (e.g., order line items inside an order document).
• Reference (store a foreign <em>id) when the related data is large, shared across many documents, or updated frequently and independently (e.g., a user</em>id reference in an order, where user profiles are queried separately).

A common pitfall is the unbounded array anti-pattern: embedding an array that grows without limit (e.g., all comments on a post) eventually hits MongoDB's 16 MB document size limit and degrades update performance. The fix is to reference comments in a separate collection.

When to Choose MongoDB

• Product catalogs with heterogeneous attributes per category.
• Content management systems where each content type has a different schema.
• Mobile/IoT applications where the schema evolves rapidly.
• Not ideal for: complex multi-entity ACID transactions, highly normalized data with many relationships, or reporting workloads better served by a columnar warehouse.

Redis: In-Memory Data Structures at Speed

Redis is often described as a cache, but that undersells it. It is an in-memory data structure server that supports strings, hashes, lists, sets, sorted sets, streams, bitmaps, and HyperLogLogs — each with O(1) or O(log n) operations. Persistence is optional (RDB snapshots, AOF log, or both). Redis is single-threaded for command execution, which eliminates lock contention and delivers predictable sub-millisecond latency.

Core Data Structures and Use Cases

import redis

r = redis.Redis(host='localhost', port=6379, decode_responses=True)

# --- STRING: cache an expensive computation result ---
r.set('report:2024-Q1:summary', 'Revenue $4.2M', ex=3600)  # TTL 1 hour
print(r.get('report:2024-Q1:summary'))

# --- HASH: store a user profile without a full serialization ---
r.hset('user:1001', mapping={
    'name': 'Alice', 'email': 'alice@example.com',
    'login_count': 0
})
r.hincrby('user:1001', 'login_count', 1)
print(r.hgetall('user:1001'))  # {'name': 'Alice', 'login_count': '1', ...}

# --- SORTED SET: real-time leaderboard ---
r.zadd('leaderboard:week42', {'alice': 9800, 'bob': 7200, 'carol': 11500})
top3 = r.zrevrange('leaderboard:week42', 0, 2, withscores=True)
print(top3)  # [('carol', 11500.0), ('alice', 9800.0), ('bob', 7200.0)]

# --- LIST: lightweight task queue ---
r.rpush('jobs:email', 'job:4421', 'job:4422')
next_job = r.blpop('jobs:email', timeout=5)  # blocking pop
print(next_job)  # ('jobs:email', 'job:4421')

Atomic Operations and Lua Scripts

Redis guarantees atomicity at the command level. For multi-step logic, use MULTI/EXEC transactions or Lua scripts (executed atomically server-side):

# Atomic increment-and-check for rate limiting
lua_script = """
local current = redis.call('INCR', KEYS[1])
if current == 1 then
    redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return current
"""
rate_limit = r.register_script(lua_script)

def check_rate_limit(user_id: str, window_sec: int = 60, max_requests: int = 100) -> bool:
    key = f'rate:{user_id}'
    count = rate_limit(keys=[key], args=[window_sec])
    return int(count) <= max_requests

print(check_rate_limit('user:1001'))  # True

Redis Pub/Sub and Streams

Redis Streams (XADD, XREAD) provide a persistent, ordered, consumer-group-aware message log — a lightweight alternative to Kafka for moderate throughput workloads.

Apache Cassandra: Wide-Column Store for Massive Write Throughput

Cassandra is a leaderless, peer-to-peer distributed database designed for write-heavy workloads across multiple data centers. It delivers linear write scalability — doubling the cluster roughly doubles write throughput. The trade-off is that queries must be designed around the primary key; arbitrary filtering or sorting is not supported without full-table scans.

Data Model: Partitions and Clustering

Cassandra organizes data into tables. Each table's primary key has two components:

• Partition key: determines which node(s) hold the data via consistent hashing. All rows with the same partition key are stored together on disk — this is the unit of locality.
• Clustering columns: sort rows within a partition; enable range queries within a partition.

The cardinal rule: design tables for queries, not for normalization. Denormalization and duplication are expected and encouraged.

from cassandra.cluster import Cluster
from cassandra.query import SimpleStatement

cluster = Cluster(['127.0.0.1'])
session = cluster.connect()

# Create keyspace and table
session.execute("""
    CREATE KEYSPACE IF NOT EXISTS iot
    WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}
""")
session.set_keyspace('iot')

session.execute("""
    CREATE TABLE IF NOT EXISTS sensor_readings (
        sensor_id  TEXT,
        ts         TIMESTAMP,
        value      DOUBLE,
        unit       TEXT,
        PRIMARY KEY (sensor_id, ts)
    ) WITH CLUSTERING ORDER BY (ts DESC)
""")

Here sensor_id is the partition key and ts is the clustering column. This layout answers the canonical IoT query — "give me the last N readings for sensor X" — as a fast partition-local range scan.

from datetime import datetime, timedelta
import uuid

# Bulk insert with prepared statement (avoids re-parsing)
insert_stmt = session.prepare(
    "INSERT INTO sensor_readings (sensor_id, ts, value, unit) VALUES (?, ?, ?, ?)"
)

import random
base_ts = datetime.utcnow()
for i in range(100):
    session.execute(insert_stmt, (
        'sensor-42',
        base_ts - timedelta(minutes=i),
        round(20 + random.gauss(0, 2), 2),
        'celsius'
    ))

# Range query — efficient because it stays within one partition
rows = session.execute("""
    SELECT ts, value FROM sensor_readings
    WHERE sensor_id = 'sensor-42'
    AND ts >= %s
    LIMIT 10
""", (base_ts - timedelta(hours=1),))

for row in rows:
    print(f"{row.ts:%H:%M:%S}  {row.value:.2f} °C")

Tunable Consistency

Cassandra's consistency level is set per query. The rule for strong consistency is:

Common levels: ONE (lowest latency, weakest guarantee), QUORUM (majority, balances both), ALL (strongest, least available).

from cassandra import ConsistencyLevel
from cassandra.query import SimpleStatement

strict_read = SimpleStatement(
    "SELECT * FROM sensor_readings WHERE sensor_id = 'sensor-42' LIMIT 1",
    consistency_level=ConsistencyLevel.QUORUM
)
result = session.execute(strict_read)

When to Choose Cassandra

• Time-series data: metrics, IoT, logs, financial tick data.
• Write-heavy workloads that must scale horizontally across multiple data centers.
• Use cases where the query pattern is known and fixed (no ad-hoc analytics).
• Not ideal for: OLAP analytics, complex joins, strong ACID transactions, or arbitrary query patterns.

Why Graphs?

Many real-world problems are fundamentally relational in the graph sense — not the table sense. Fraud detection, recommendation engines, knowledge graphs, social networks, and supply-chain analysis all require traversing chains of relationships. In a relational database, each hop in a multi-step path requires a JOIN. For a 6-hop traversal over millions of nodes, the query plan can involve joins so expensive they become impractical. A native graph database stores relationships as first-class physical pointers, making traversal O(degree) per step rather than O(log n) per join.

The Property Graph Model

Neo4j uses the labeled property graph model:

• Nodes represent entities. Each node has one or more labels (e.g., Person, Movie) and a set of key-value properties.
• Relationships (edges) connect two nodes, have a type (e.g., ACTED<em>IN, FOLLOWS), a direction, and can also carry properties (e.g., {role: "Neo"}).

This is more expressive than a simple adjacency list because both nodes and edges carry structured data.

Cypher Query Language

Cypher is Neo4j's declarative query language. Its syntax uses ASCII art to represent graph patterns:

(node)-[:RELATIONSHIP]->(other_node)

Round parentheses denote nodes; square brackets with a colon denote relationship types; arrows indicate direction.

Setup with the Python Driver

from neo4j import GraphDatabase

URI = "bolt://localhost:7687"
AUTH = ("neo4j", "password")

driver = GraphDatabase.driver(URI, auth=AUTH)

def run_query(query: str, params: dict = None):
    with driver.session() as session:
        return list(session.run(query, params or {}))

# Create nodes and relationships
run_query("""
    MERGE (alice:Person {name: 'Alice', age: 34})
    MERGE (bob:Person   {name: 'Bob',   age: 28})
    MERGE (carol:Person {name: 'Carol', age: 41})
    MERGE (neo4j:Technology {name: 'Neo4j'})
    MERGE (python:Technology {name: 'Python'})

    MERGE (alice)-[:KNOWS {since: 2020}]->(bob)
    MERGE (bob)-[:KNOWS   {since: 2021}]->(carol)
    MERGE (alice)-[:USES]->(neo4j)
    MERGE (alice)-[:USES]->(python)
    MERGE (bob)-[:USES]->(python)
""")
print("Graph seeded")

Pattern Matching and Traversal

# Find all people Alice knows (1 hop)
results = run_query("""
    MATCH (a:Person {name: 'Alice'})-[:KNOWS]->(friend:Person)
    RETURN friend.name AS name, friend.age AS age
""")
for r in results:
    print(r['name'], r['age'])
# Bob 28

# Friends of friends (2 hops) — not yet in Alice's direct network
results = run_query("""
    MATCH (a:Person {name: 'Alice'})-[:KNOWS*2]->(fof:Person)
    WHERE NOT (a)-[:KNOWS]->(fof)
    RETURN DISTINCT fof.name AS name
""")
print([r['name'] for r in results])  # ['Carol']

# Variable-length path: anyone reachable within 1-3 KNOWS hops
results = run_query("""
    MATCH p = (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(other:Person)
    RETURN other.name AS name, length(p) AS hops
    ORDER BY hops
""")
for r in results:
    print(f"{r['name']} ({r['hops']} hops)")

Shortest Path

# Built-in shortestPath function
result = run_query("""
    MATCH p = shortestPath(
        (a:Person {name: 'Alice'})-[:KNOWS*]-(c:Person {name: 'Carol'})
    )
    RETURN [node in nodes(p) | node.name] AS path, length(p) AS hops
""")
print(result[0]['path'])  # ['Alice', 'Bob', 'Carol']
print(result[0]['hops'])  # 2

PageRank and Graph Algorithms (GDS)

Neo4j's Graph Data Science (GDS) library provides in-database implementations of PageRank, Louvain community detection, Betweenness Centrality, Node2Vec, and dozens of other algorithms. The workflow is: project a subgraph into memory → run the algorithm → write results back to nodes.

# Project graph, run PageRank, read back centrality scores
run_query("""
    CALL gds.graph.project('social', 'Person', 'KNOWS')
""")

results = run_query("""
    CALL gds.pageRank.stream('social')
    YIELD nodeId, score
    RETURN gds.util.asNode(nodeId).name AS name,
           round(score, 4) AS pagerank
    ORDER BY pagerank DESC
""")
for r in results:
    print(f"{r['name']:10s}  {r['pagerank']}")

run_query("CALL gds.graph.drop('social')")

Data Modeling for Graphs

Effective graph modeling requires different intuitions than relational modeling:

• Make relationships explicit and typed. Instead of a junction table user</em>follows_user, use a (:Person)-[:FOLLOWS]->(:Person) relationship. Relationship properties can carry weight, timestamp, or context.
• Avoid super-nodes. A node with millions of relationships (e.g., a popular hashtag) becomes a traversal bottleneck. Consider partitioning or using intermediate nodes.
• Choose labels for query optimization. Labels are indexed and allow Neo4j to constrain the starting set of nodes before traversal.

When to Choose Neo4j

• Fraud detection: detecting rings of connected fraudulent accounts.
• Recommendation: "users who bought X also bought Y" via common purchase relationships.
• Knowledge graphs and ontologies.
• Network and IT dependency analysis.
• Not ideal for: bulk analytical aggregations over all nodes/edges, or simple CRUD with no meaningful relationships.

Embeddings and the Rise of Vector Search

Large language models (LLMs) and embedding models encode text, images, and other data as dense vectors in a high-dimensional space (commonly 768 to 3072 dimensions) such that semantically similar items are geometrically close. This enables a query mode that relational and key-value stores cannot support: semantic similarity search — find the $k$ documents most similar in meaning to a query, even when no keyword matches exist.

Vector databases are purpose-built to store, index, and query these embedding vectors at scale. They are the infrastructure layer that makes Retrieval-Augmented Generation (RAG) practical.

Approximate Nearest Neighbor (ANN) Search

Exact nearest-neighbor search in $d$ dimensions requires computing the distance from the query vector to every stored vector — O(n·d) cost. For millions of vectors this is too slow. Vector databases use Approximate Nearest Neighbor (ANN) algorithms that trade a small accuracy loss for orders-of-magnitude speedup:

• HNSW (Hierarchical Navigable Small World): builds a multi-layer graph; navigation starts at the top layer and greedily descends. Used by Chroma, Weaviate, pgvector.
• IVF (Inverted File Index): clusters vectors using k-means; at query time only the nearest clusters are searched. Used by FAISS, Pinecone.
• LSH (Locality-Sensitive Hashing): projects vectors into buckets where similar vectors are likely to collide; rarely used in production anymore.

The similarity metric is typically cosine similarity or dot product for normalized embeddings:

ChromaDB: Local-First Vector Store

Chroma is an open-source vector database designed for ease of use — ideal for prototyping and small-to-medium scale RAG applications.

import chromadb
from chromadb.utils import embedding_functions

# Use a local sentence-transformer model for embeddings
embedder = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name='all-MiniLM-L6-v2'
)

client = chromadb.Client()  # in-memory; use PersistentClient for disk
collection = client.create_collection(
    name='docs',
    embedding_function=embedder,
    metadata={'hnsw:space': 'cosine'}
)

# Index a small knowledge base
corpus = [
    "Redis is an in-memory data structure store used as a cache.",
    "Cassandra is a wide-column store designed for high write throughput.",
    "Neo4j is a native graph database using the Cypher query language.",
    "MongoDB stores data as flexible JSON-like BSON documents.",
    "Vector databases enable semantic similarity search over embeddings.",
    "The CAP theorem states a distributed system can guarantee at most two of C, A, P.",
]

collection.add(
    documents=corpus,
    ids=[f'doc{i}' for i in range(len(corpus))]
)

# Semantic query
results = collection.query(
    query_texts=["Which database is good for caching?"],
    n_results=2
)
for doc, dist in zip(results['documents'][0], results['distances'][0]):
    print(f"[dist={dist:.4f}] {doc[:70]}")

Building a RAG Pipeline

RAG augments an LLM's context window with relevant retrieved documents, enabling accurate responses about private or recent knowledge that the model was not trained on. The architecture has three phases:

1. Indexing (offline): chunk documents, generate embeddings, store in vector DB.
2. Retrieval (online): embed the user query, ANN-search the vector DB, return top-k chunks.
3. Generation (online): pass query + retrieved context to the LLM as a prompt.

import openai
import chromadb
from chromadb.utils import embedding_functions

# --- INDEXING PHASE ---
embedder = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name='all-MiniLM-L6-v2'
)
client = chromadb.PersistentClient(path='./rag_db')
collection = client.get_or_create_collection('knowledge', embedding_function=embedder)

docs = [
    {"id": "d1", "text": "Redis Cluster uses hash slots (0-16383) to partition keys.",
     "source": "redis-docs"},
    {"id": "d2", "text": "Cassandra's write path: commit log -> memtable -> SSTable flush.",
     "source": "cassandra-docs"},
    {"id": "d3", "text": "Neo4j APOC library provides utility procedures for graph algorithms.",
     "source": "neo4j-docs"},
]
collection.upsert(
    ids=[d['id'] for d in docs],
    documents=[d['text'] for d in docs],
    metadatas=[{'source': d['source']} for d in docs]
)

# --- RETRIEVAL + GENERATION PHASE ---
def rag_query(question: str, k: int = 2) -> str:
    retrieved = collection.query(query_texts=[question], n_results=k)
    context = "\n".join(retrieved['documents'][0])

    prompt = f"""Answer the question using ONLY the context below.

Context:
{context}

Question: {question}"""

    # Requires OPENAI_API_KEY environment variable
    response = openai.chat.completions.create(
        model='gpt-4o-mini',
        messages=[{'role': 'user', 'content': prompt}]
    )
    return response.choices[0].message.content

print(rag_query("How does Cassandra handle writes?"))

Metadata Filtering

Vector databases support hybrid filtering: combine ANN search with exact metadata predicates to constrain results to a subset of documents.

# Only retrieve from 'redis-docs' source
results = collection.query(
    query_texts=["How does partitioning work?"],
    where={"source": "redis-docs"},
    n_results=2
)

pgvector: Vector Search in PostgreSQL

For teams already running PostgreSQL, pgvector adds a native vector column type and ANN index — enabling vector search without an additional service.

import psycopg2
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')

conn = psycopg2.connect("dbname=mydb user=postgres")
conn.autocommit = True
cur = conn.cursor()

cur.execute("CREATE EXTENSION IF NOT EXISTS vector")
cur.execute("""
    CREATE TABLE IF NOT EXISTS embeddings (
        id   SERIAL PRIMARY KEY,
        text TEXT,
        vec  vector(384)
    )
""")
cur.execute("CREATE INDEX IF NOT EXISTS vec_idx ON embeddings USING hnsw (vec vector_cosine_ops)")

texts = ["Machine learning models require training data.",
         "SQL joins combine rows from multiple tables."]
for text in texts:
    vec = model.encode(text).tolist()
    cur.execute("INSERT INTO embeddings (text, vec) VALUES (%s, %s)", (text, vec))

# Semantic search using pgvector's <=> cosine distance operator
query_vec = model.encode("How do neural networks learn?").tolist()
cur.execute("""
    SELECT text, 1 - (vec <=> %s::vector) AS similarity
    FROM embeddings
    ORDER BY vec <=> %s::vector
    LIMIT 2
""", (query_vec, query_vec))

for text, sim in cur.fetchall():
    print(f"{sim:.4f}  {text}")

Choosing a Vector Store

• Chroma: easiest to self-host; great for prototyping and single-machine scale.
• Pinecone: fully managed, production-grade; no infrastructure management; serverless tier available.
• Weaviate: open-source, multi-modal, built-in BM25 hybrid search.
• pgvector: best when you already operate PostgreSQL and scale needs are moderate (up to ~1M vectors before dedicated stores become preferable).
• Qdrant: open-source, Rust-based, excellent filter performance and quantization support.