Module semantic 

A precomputed, quantized semantic index.

SemanticFallback says what the engine needs — ids in relevance order — and nothing about how to get them. This is the in-process implementation, built once from the corpus and searched without a network call.

Why it is precomputed

Embedding is a network round trip: 259 ms at p50, measured, flat across widths (tests/serving_latency_probe.rs). The interactive budget is 10 ms and the speculative one 100 ms, so embedding at recall time is not a tuning problem — it is three and a half orders of magnitude out.

Document vectors therefore have to exist before the question does. Ingestion embeds each record once, concurrently — the same probe measured 88 embeds/s at ×32, so a 16,000-record corpus is about three minutes of wall-clock, not an overnight job — and PrecomputedSemanticIndex holds the result.

The query embedding is the round trip that remains, and this type does not pretend otherwise: it takes an Embedder, and whether that fits the caller’s budget is the caller’s architecture decision. A local model fits; a remote one is for the speculative path, where nobody is waiting, and even then only if the budget is raised past 259 ms. See PrecomputedSemanticIndex::search.

Why it is quantized

An exact float32 scan over 16,000 records takes 15.2 ms — past the 10 ms interactive budget on its own, before the query is even embedded. Packing each vector to one bit per dimension and scoring with XOR and popcount takes 812 µs, and reranking the top 50 against the float vectors restores the exact ranking: 105 µs against 1 ms at 1,199 records, and identical top-1, top-5 and MRR (tests/quantization_probe.rs).

configuration	fused top-5	RAM at 16k	scan at 16k
float32 exact	79/93	49 MB	15.2 ms
1-bit packed	77/93	2 MB	812 µs
1-bit + exact rerank	78/93	2 MB + floats	1.3 ms

The quality differences across that table are one or two questions out of 93 — noise. The cost differences are 24× in memory and 12× in scan time, which are not. Priced out, that is $0.021 per user per month against $0.158.

The float vectors are kept for the rerank. A deployment that cannot afford them resident can drop to PrecomputedSemanticIndex::without_rerank and lose about one question in 93, or hold them on SSD and fault in fifty per query.

Structs

OkfVectorStore

Vectors kept beside the records, in the same store the OKF Markdown uses.

PrecomputedSemanticIndex

A semantic index built ahead of time and searched in process.

StaticEmbedder

An embedder backed by a fixed table, for tests and offline replay.

Constants

RERANK_DEPTH

How many candidates the quantized scan proposes before the exact rerank.

Traits

Embedder

Turns text into a vector.

VectorStore

Somewhere to keep vectors between processes.