Combining full text and semantic search
Related to the
find_command
project. (Why doesn’t search_chroma.py show up in the GitHub repo?)
There are two things to consider when combining records from full-text and semantic search: which records get considered, and how those records are ordered.
Full-text candidates → semantic ranking #
FTS5 finds candidates → semantic similarity orders those candidates. For example, require window to appear in the candidates, then rank those descriptions by similarity to “close the current window”.
Chroma supports restricting a query to specified record IDs, so the IDs returned by FTS5 can be passed to Chroma Chroma query API .
The limitation with full-text candidates → semantic ranking is that semantic ranking can’t recover anything excluded by FTS5. E.g., requiring close AND window would exclude “kill the current window”, even though it’s semantically relevant. The FTS → semantic ranking approach makes sense when excluding or including some results based on a key word is a genuine requirement.
Independent searches → fusion #
Rank the FTS5 results and combine with the (ranked by design) semantic results. The combined ranking is used to decide what gets included in the final results. With this approach, each method can contribute records that the other method missed.
Common ways of combining the results:
- Reciprocal Rank Fusion (RRF): combines each record’s position in the result list. Appearning near the top of either list helps; appearing in both results helps more.
- Weighted score combination: combines lexical and semantic scores, with adjustable weights. The different scales and directions of the scores needs handling: Elastic hybrid-search overview .
RRF is a useful first experiment, because it doesn’t require comparable raw scores. The Elastic docs go into a bit of detail: RRF documentation .
For RRF, the FTS5 results will need relevance ranking. The current ID ordering isn’t meaningful input to RRF. BM25 ranking can be used for this. See Create FTS5 virtual table in the SQLite CLI#Ranking results with the BM25 algorithm and The bm25() function .
Combine candidates → a separate reranker #
FTS5 candidates + semantic candidates
↓
merge and deduplicate
↓
score with a reranker
A cross-encoder reranker processes the query and a candidate description together, producing a relevance score for that pair. See Sentence Transformers: retrieve and rerank .
Other variations #
- Reverse the sequence: semantic candidates first, then lexical filtering and ranking.
- Fallback: try full-text search first; use semantic search when it returns too few results.
- Query expansion: add synonyms or alternate wording before retrieval—e.g., searching for both “close” and “kill”.
- Explicit constraints plus semantic intent: e.g., require
window, but use “close the current window” as the semantic query. This separates mandatory terms from descriptive language.
Next steps #
Try:
- BM25-ranked FTS5 to get a baseline
- FTS5 candidates → semantic ranking
- Independent retrieval → RRF
- Cross-encoder reranking of combined FTS5 and semantic candidates