Retrieval pipeline¶

Step by step, what GET /api/context/{project}/relevant does.

flowchart TB
  IN["GET .../relevant?query&limit"] --> S1
  S1["1 · resolve project key → internal id"] -->|"unknown → 404"| S2
  S2["2 · load candidates<br/>newest first, ≤ MAX_CANDIDATES"] --> S3
  S3["3 · filter: row.org_id == caller.org_id"] --> S4
  S4{"4 · candidates empty?"} -->|yes| OUT0["return []"]
  S4 -->|no| S5
  S5{"5 · query given?"} -->|yes| S6a["embed(query)"]
  S5 -->|no| S6b["similarity = 1.0 for all"]
  S6a --> S7
  S6b --> S8
  S7["7 · per candidate:<br/>stored embedding compatible?<br/>yes → use · no → recompute"] --> S8
  S8["8 · score = 0.40·sim + 0.20·rec + 0.25·imp + 0.15·freq"] --> S9
  S9["9 · sort desc, take limit"] --> S10
  S10["10 · strip embedding, attach justification"] --> OUT["memories[]"]

The steps¶

1 — Project resolution¶

Your project key maps to an internal proj_… id. This read path is strict: an unknown key is a 404, not an empty result. (The write path is the opposite — it creates.)

2 — Candidate generation¶

Every memory in the project, newest first, up to VENKAI_RETRIEVAL_MAX_CANDIDATES (default 10000). No index, no pre-filter, no similarity threshold at the database.

When the cap is hit, the server increments a counter and logs:

retrieve: candidate set truncated at VENKAI_RETRIEVAL_MAX_CANDIDATES=10000
for project proj_… — memories older than the newest 10000 are not being
scored and cannot be retrieved. Raise the ceiling or shard the project.

The verbosity is intentional. This is the one place recall can be lost without an error, and a previous 500-row cap hid there for a long time precisely because it said nothing.

3 — Organization re-check¶

Candidates were already loaded through an org-keyed project. Every row is checked again against the caller's org_id before it can reach a result. Redundant on purpose: one missed WHERE clause upstream cannot become a cross-tenant leak.

5–7 — Similarity¶

With a query: embed it once, then cosine against each candidate.

A stored embedding is used when its length matches the provider's dimension. When it does not — provider changed, or the row predates embedding persistence — it is recomputed for this request rather than dropping the memory. The recomputation is not written back, so it repeats on every query until backfilled (Embeddings).

Without a query, every candidate gets similarity = 1.0 and no embedding work happens at all.

8 — Scoring¶

score = (0.40 * similarity
       + 0.20 * 0.5 ** (age_days / 14.0)
       + 0.25 * importance
       + 0.15 * frequency_score(access_count))

All four terms are in [0, 1], so score is too. Weights are global constants, not per-request parameters.

9 — Selection¶

Sort descending, slice to limit. No threshold. Nothing is dropped for scoring badly — limit is the only filter, so a sparse project returns its least-bad matches without comment.

10 — Response shaping¶

embedding is stripped. The internal project id is swapped back for your key. score, similarity and justification are attached.

What is deliberately absent¶

Retrieval does not write¶

access_count, last_accessed_at and updated_at are untouched. The frequency term therefore stays near zero for most memories, and that is preferable to the alternative — see Ranking.

No caching¶

Every call runs the full pipeline. Identical consecutive queries do identical work. Since there is no model call, the cost is database I/O plus arithmetic, and a cache would add an invalidation problem for a modest saving.

No query expansion, no reranking¶

The query is used exactly as given: no synonym expansion, no LLM rewriting, no cross-encoder second pass. One embedding, one linear blend, one sort. That is what makes the endpoint deterministic and free of model cost — and also what caps its ceiling.

Complexity¶


Candidate load	one query, `O(n)` rows
Query embedding	one, or zero without a query
Similarity	`O(n · d)` — `d` is 128 or 384
Recomputation	`O(stale · embed_cost)` — zero on a healthy store
Sort	`O(n log n)`
Total	linear in project size

The read path scales with memories per project, not with request volume. That is the ceiling to watch.

Tuning¶

Symptom	Lever
Truncation warnings	Raise `VENKAI_RETRIEVAL_MAX_CANDIDATES`, or split the project
Paraphrases not matching	`VENKAI_EMBEDDING_PROVIDER=semantic`
Latency growing	Check project size; backfill stale embeddings
Irrelevant results returned	Apply a `score` floor client-side — there is none server-side
Wrong things ranked first	Raise `importance` at write time; it is 25 % of the score