Hybrid Search with Keyword Plus Embeddings

Embeddings are good at paraphrase and bad at exact tokens. "pg_hba.conf", "SIGTERM", and an error code from your logs are keyword problems. Pure vector search will happily return a nearby essay about authentication while missing the one line that names the file. Hybrid search runs a keyword index and an embedding scan, then fuses the two ranked lists. On one VPS that is FTS5 or tsvector plus the float32 table you already own.

Why fusion beats a bigger model

A larger encoder will not learn that users type flags, filenames, and HTTP status codes. Those strings need an inverted index. Reciprocal rank fusion (RRF) is enough: score = 1 / (k + rank_keyword) + 1 / (k + rank_vector) with k around 60. You do not need a learned ranker until you have click logs. RRF has no training set and fails open if one list is empty.

Keep both retrievals cheap. Keyword search should hit an index, not LIKE percent. Vector search should filter to the same language or path prefix first, then scan. Returning 40 from each list and fusing to 12 is a solid default before reranking.

CREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts USING fts5(
  path,
  content,
  content='chunks',
  content_rowid='id'
);

CREATE TRIGGER chunks_ai AFTER INSERT ON chunks BEGIN
  INSERT INTO chunks_fts(rowid, path, content)
  VALUES (new.id, new.path, new.content);
END;

Match quoted phrases when the query looks like an identifier. A simple heuristic: if the query has a dot, underscore, or mixed case token, require that token in FTS. Let the vector side still run so a paraphrased question is not empty.

Postgres tsvector when SQLite is not the writer

If the store is Postgres, add a generated tsvector column on content and a GIN index. websearch_to_tsquery is friendlier than raw tsquery for user text. Keep the same RRF in the application. Do not try to express RRF inside one giant SQL statement on day one. Two queries and a dict in Python is inspectable.

from collections import defaultdict

def rrf(keyword_ids, vector_ids, k=60, limit=12):
    scores = defaultdict(float)
    for rank, row_id in enumerate(keyword_ids, start=1):
        scores[row_id] += 1.0 / (k + rank)
    for rank, row_id in enumerate(vector_ids, start=1):
        scores[row_id] += 1.0 / (k + rank)
    ranked = sorted(scores.items(), key=lambda item: item[1], reverse=True)
    return [row_id for row_id, _score in ranked[:limit]]

Log three things per query: the FTS string you actually ran, the top 5 vector ids, and the fused ids. When a user says search is wrong, that line tells you which leg failed. Bind both the API and the database to localhost. L3/L4 DDoS filtering on the public HTTP port is not a substitute for leaving FTS on the internet.

Scoring details that actually move nDCG

Normalize vectors. If one model is not L2-normalized, cosine is wrong and fusion will trust the wrong leg. BM25 from FTS5 is already a rank; do not mix raw BM25 with cosine in a linear weighted sum until you tune on labeled queries. RRF ignores the raw magnitudes, which is a feature when the two scales differ.

Strip stop words on the keyword side only. The embedding query should be the original question. "How do I stop a systemd unit" needs "stop" and "unit" for MiniLM. FTS can drop "do" and "I".

If the keyword list is empty, return vectors only. If the vector worker is down, return FTS only and set a header. Hybrid should degrade, not 500.

On Intel Xeon Platinum with High-Speed SSD, FTS5 over a few hundred MB of text is sub-millisecond. The vector scan dominates. Filter first. A docs-only path prefix turns a 80k scan into 8k. That filter belongs in SQL or in the SQLite WHERE, not in Python after you have already copied every blob into RAM.

Evaluation without a research bench

Take 30 questions from real tickets. Mark one or more good chunk ids. Measure recall@10 for FTS alone, vectors alone, and RRF. If RRF loses to FTS, your vector model or chunking is the bug, not fusion. If RRF loses to vectors, your FTS query is over-tokenized. Stay here until recall@10 is boring. Rerankers amplify whatever you feed them.

Tune k in RRF only after that set is stable. Raising k toward 100 flattens both lists and hides a broken FTS query. Lowering k toward 10 makes the first keyword hit almost unbeatable, which is correct for error codes and wrong for a paraphrased how-to. Keep k at 60 until the 30 questions are boring. Then change one thing at a time: k, the FTS tokenizer, or the vector model. Never all three on a Friday.

Store the 30 questions in git. Run them after ingest. Hybrid search is an ops surface: indexes, triggers, and a fusion function. Treat it like that, not like a demo widget.

Takeaway

Run keyword and vector retrieval side by side, fuse with RRF, and log both lists. Exact tokens and paraphrase are different jobs. You can spin up an Ubuntu 24.04 instance on Netbay in Lucknow in under 60 seconds and put FTS5 next to your float32 table on the same disk — netbayhosts.in.