Skip to content

Context selection

This is the page to read if you are evaluating Venkai seriously.

Retrieval is not selection

flowchart LR
  subgraph RET["Retrieval — 'what matches?'"]
    A["candidates"] --> B["similarity"] --> C["ranked list"]
  end
  subgraph SEL["Context selection — 'what ships?'"]
    C --> D["budget"] --> E["redundancy"] --> F["diversity"] --> G["ordering"] --> H["final context"]
  end

A vector database answers the left box: give me the nearest neighbours. Answering the right box is a different job, because the constraint is not "what is similar" but "what fits in the prompt, without repeating itself, and in what order".

Venkai's /relevant endpoint spans both boxes — but not all of the right one. Being precise about which parts is the point of this page.

What Venkai implements today

Stage Status How
Candidate generation Stable Every memory in the project, newest first, capped at VENKAI_RETRIEVAL_MAX_CANDIDATES
Semantic similarity Beta Cosine over stored embeddings; genuinely semantic only with the semantic provider
Metadata scoping Stable Organization + project; type/agent filters on the list endpoint
Importance Stable 25 % of score, supplied at write time
Recency Stable 20 % of score, 14-day half-life
Access frequency Experimental 15 % of score, but ~0 in practice — see Ranking
Ranking Stable Fixed linear blend, deterministic
Filtering by threshold Planned No minimum-score cutoff exists. limit=10 on a project with 10 memories returns all 10, however irrelevant.
Context budget in tokens Planned The budget knob is limit, a count of memories, not tokens.
Redundancy removal Planned Two near-identical memories both ship. No MMR, no dedup.
Diversity / type balancing Planned Nothing guarantees a mix of constraints and facts.
Ordering for the prompt Planned Returned in score order. Lost-in-the-middle placement is your call.

Read that table as the honest boundary. Venkai gives you a ranked, explained, bounded list. It does not yet give you a packed, deduplicated, budget-aware context block.

What that means in practice

Three things you must handle client-side today.

1. limit is a count, not a budget

Ten memories of 40 characters and ten of 8000 both satisfy limit=10. If your prompt budget is in tokens, enforce it yourself:

def select(memories, max_chars=4000):
    """Take ranked memories until the character budget is spent.
    Venkai ranks; the budget is the caller's to enforce."""
    out, used = [], 0
    for m in memories:
        cost = len(m["content"])
        if used + cost > max_chars:
            continue          # skip, don't break — a short high-scorer may still fit
        out.append(m)
        used += cost
    return out

2. There is no relevance floor

Every result has a score, and nothing is dropped for scoring badly. On a sparse project, /relevant will hand you the least-bad matches with a straight face. Apply your own cutoff:

memories = [m for m in resp["memories"] if m["score"] >= 0.35]

Choose the threshold empirically against your own data — and note that with the hashing provider, absolute score values sit low across the board, so a threshold tuned on hashing will not transfer to semantic.

3. Near-duplicates both ship

If your agent writes "we chose Postgres" on three consecutive runs, all three can occupy your top-5. Deduplicate before writing (Memory), or collapse on the way out.

Why the ranking blend is the interesting part

The reason Venkai does not just return nearest neighbours: similarity alone is a bad selection policy for agent memory.

A hard constraint written six months ago is more relevant to today's task than a chatty fact written this morning that happens to share vocabulary with the query. Pure similarity ranks the chatty fact first. The importance and recency terms exist to encode "this mattered" and "this is current" as first-class signals rather than hoping the embedding notices.

That is a policy, and it is the thing Venkai ships that a vector index does not. Its weights are fixed today — inspectable in Ranking, overridable never. If your domain needs a different blend, that is a real limitation and you should say so.

What we do not claim

No measurement is presented here showing that this policy beats plain similarity, or beats stuffing the whole history into the prompt. No such benchmark has been published. Evaluation explains what exists internally and why it is not quoted.

Evaluate the policy against your own workload. The scores and justifications are on every result specifically so you can.