Skip to content

Retrieval

Retrieval is finding candidates and scoring them. Deciding what actually goes into the prompt is context selection — a related but separate question.

The pipeline

flowchart TB
  Q["GET /api/context/{project}/relevant?query=…&limit=N"] --> R1
  R1["1. Resolve project key → internal id<br/><i>unknown key → 404</i>"] --> R2
  R2["2. Load candidates<br/><i>newest first, capped at MAX_CANDIDATES</i>"] --> R3
  R3["3. Re-verify org ownership<br/><i>defence in depth</i>"] --> R4
  R4{"query given?"}
  R4 -->|yes| R5["4a. Embed query, cosine vs each candidate"]
  R4 -->|no| R6["4b. similarity = 1.0 for all"]
  R5 --> R7
  R6 --> R7
  R7["5. Blend: relevance · recency · importance · frequency"] --> R8
  R8["6. Sort by score, take top N"] --> R9
  R9["7. Strip embeddings, attach justification"] --> OUT["memories[]"]

Every step is deterministic. No model is called. The same store and the same query produce the same list, every time.

Candidate generation

Candidates are every memory in the project, newest first, up to VENKAI_RETRIEVAL_MAX_CANDIDATES (default 10000). There is no ANN index and no pre-filter — scoring is a Python loop over that set.

Consequences worth knowing:

  • Cost is O(candidates) per query, not O(log n). At 10 000 memories and a 128-d hashing vector this is milliseconds; at 384-d with a large project it is measurably slower.
  • The cap truncates by recency, so if you exceed it the memories you lose are the oldest ones — which may be exactly your foundational constraints.
  • Truncation is never silent: the server emits a warning naming the project and the ceiling, and increments a counter. If you run self-hosted, alert on it.

Raise the ceiling, or split into more projects, before you hit it.

Similarity

With a query, each candidate is compared by cosine similarity between the query embedding and the memory's stored embedding.

If a stored embedding is missing or has the wrong dimension — because the provider changed, or the row predates embedding persistence — it is recomputed on the fly for that request rather than dropping the memory from the results. Correct, but slow: a project full of stale embeddings re-embeds on every query until it is backfilled.

Without a query, similarity is 1.0 for everything and the ranking is recency + importance + frequency alone.

Scoring

score = 0.40 · similarity
      + 0.20 · recency        exp decay, 14-day half-life
      + 0.25 · importance     as supplied at write time
      + 0.15 · frequency      log-scaled access_count

Full treatment, including why the weights are what they are: Ranking.

The two read endpoints

GET /api/context/{project} GET /api/context/{project}/relevant
Ordering created_desc (configurable) by score
Filters type, q, agent, offset none
Scoring none full blend
Page cap 200 none — limit is respected as given
Returns context[], decisions[], count, total, offset memories[]
Use for dashboards, audit, export prompts

q on the list endpoint is a substring filter, not a ranking signal. It is not a cheap /relevant.

Latency

There is no published latency benchmark for Venkai, and this documentation does not quote one. What you can reason about from the implementation:

  • Per query: one candidate load, n cosine computations over d dimensions, one sort. Plus one embedding of the query.
  • hashing (128-d) is cheap enough to ignore. semantic (384-d MiniLM) costs a model forward pass per query, and per candidate whose embedding is stale.
  • Measure it on your own data before designing around a number.