Search Modes¶

Every ranked search this library does, local or live, uses one of three ranking strategies, chosen via mode (or --mode/-m on the CLI): "lexical", "semantic", or "hybrid". This page explains what each one actually does, so Local Search and Cache and the CLI's --mode flag make sense as more than just three interchangeable options.

Lexical: matching words¶

The default. Needs nothing beyond the base install.

Lexical search ranks by bm25, a full-text ranking algorithm, over the term, definition, and topic text actually stored locally. It only ever matches words that are actually present (or close misspellings of them). Searching "rock that holds fluid" under lexical mode will not find a term like "porous" unless those specific words appear somewhere in its stored definition.

Live search uses a related but simpler technique, since there's no whole result set to rank against ahead of time. It calculates the plain token overlap between your query and each candidate term/topic, scored as results stream in one page at a time.

Semantic: matching meaning¶

Needs the semantic extra installed (uv add "slb-glossary[semantic]"), and terms already embedded first.

Semantic search compares embeddings: numeric vectors that capture a phrase's meaning, produced by a small local model (minishlab/potion-retrieval-32M, via model2vec), downloaded once and cached, with no network call needed per query afterward. Two phrases with similar meanings end up with similar vectors even if they do not share any words. Searching "rock that holds fluid" surfaces "porous" this way, since the two land close together in vector space (measured by cosine similarity), despite sharing no words at all.

This only works on terms you've already run through embed_terms:

await slb.local.embed_terms(db)  # embeds everything not already embedded

From the CLI, the equivalent is slb local embed - see Local Cache and Sync.

embed_terms is a one-time (or periodic) cost, separate from ordinary syncing. Syncing fetches and stores terms, embed_terms computes and stores their vectors. Run it again after a sync that added new terms, with only_missing=True (the default) so it only pays for what's actually new.

Semantic scores aren't on the same scale as lexical scores

Lexical (and hybrid) scores are calibrated to roughly [0.0, 1.0]. Semantic search's cosine-similarity scores aren't calibrated the same way, which matters if you are pairing mode="semantic" with source=Source.AUTO's relevance_threshold as that threshold will be compared against an uncalibrated number in that combination. mode="hybrid" is better paired with Source.AUTO for exactly this reason.

Live search has no semantic mode at all. There's no local embedding table to compare against for a page that was just fetched, so semantic (and hybrid) ranking is local-only.

Hybrid: both, fused¶

Same requirements as semantic: the extra installed, and terms embedded.

Hybrid search runs both lexical and semantic search over the same query, then fuses their two rankings with weighted Reciprocal Rank Fusion (RRF): each result's score is based on where it ranked in each list, not the raw scores themselves, which sidesteps the lexical/semantic scale mismatch entirely. A result that ranks well in either list (or both) surfaces near the top; the fused scores are then min-max normalized back into a [0.0, 1.0]-ish band, so relevance_threshold behaves sensibly again.

This is generally the best-ranking mode once you've embedded your terms, and the recommended pairing with Source.AUTO. It's not the library-wide default ("lexical" is) specifically so that a database that's never had embed_terms run on it keeps working out of the box, without the semantic extra being forced on every install.

Choosing a mode¶

	Needs	Matches	Works live	Good `Source.AUTO` pairing
`lexical`	Nothing extra	Exact words (or near-misspellings, with `fuzzy=True`)	Yes	Yes
`semantic`	`semantic` extra + `embed_terms`	Meaning, not exact words	No (local only)	Only with care, see the scale warning above
`hybrid`	`semantic` extra + `embed_terms`	Both, fused by rank	No (local only)	Yes, generally the best default once embedded

await slb.local.search(db, "porosity", mode="lexical")  # default, exact-word match
await slb.local.search(db, "rock that holds fluid", mode="semantic")  # paraphrase match
await slb.local.search(db, "reservoir rock", mode="hybrid")  # both, fused

slb search porosity --local --mode hybrid

Where this shows up¶

slb_glossary.local.search, and the standalone lexical_search/vector_search/hybrid_search functions it dispatches to.
slb_glossary.query.search's mode parameter, with the live-fallback restriction that a live fetch can not be scored "hybrid".
The CLI's search --mode/local search --mode, and slb local export --query ... --mode.