Skip to content

Context selection

This is the page to read if you are evaluating Venkai seriously.

Retrieval is not selection

flowchart LR
  subgraph RET["Retrieval — 'what matches?'"]
    A["candidates"] --> B["similarity"] --> C["ranked list"]
  end
  subgraph SEL["Context selection — 'what ships?'"]
    C --> D["budget"] --> E["redundancy"] --> F["diversity"] --> G["ordering"] --> H["final context"]
  end

A vector database answers the left box: give me the nearest neighbours. Answering the right box is a different job, because the constraint is not "what is similar" but "what fits in the prompt, without repeating itself, and in what order".

Venkai's /relevant endpoint spans both boxes — but not all of the right one. Being precise about which parts is the point of this page.

What Venkai implements today

Stage Status How
Candidate generation Stable The memories of the project, within your organization
Semantic relevance Beta See how relevance behaves today
Metadata scoping Stable Organization + project; type/agent filters on the list endpoint
Importance Stable Supplied at write time, never decays
Recency Stable Fades gradually over weeks
Usage Experimental Not fed by retrieval — see Ranking
Ranking Stable Deterministic, explained per result
Filtering by threshold Planned No minimum-score cutoff exists. limit=10 on a project with 10 memories returns all 10, however irrelevant.
Context budget in tokens Planned The budget knob is limit, a count of memories, not tokens.
Redundancy removal Planned Two near-identical memories both ship.
Diversity / type balancing Planned Nothing guarantees a mix of constraints and facts.
Ordering for the prompt Planned Returned in score order. Lost-in-the-middle placement is your call.

Read that table as the honest boundary. Venkai gives you a ranked, explained, bounded list. It does not yet give you a packed, deduplicated, budget-aware context block.

What that means in practice

Three things you must handle client-side today.

1. limit is a count, not a budget

Ten memories of 40 characters and ten of 8000 both satisfy limit=10. If your prompt budget is in tokens, enforce it yourself:

def select(memories, max_chars=4000):
    """Take ranked memories until the character budget is spent.
    Venkai ranks; the budget is the caller's to enforce."""
    out, used = [], 0
    for m in memories:
        cost = len(m["content"])
        if used + cost > max_chars:
            continue          # skip, don't break — a short high-scorer may still fit
        out.append(m)
        used += cost
    return out

2. There is no relevance floor

Every result has a score, and nothing is dropped for scoring badly. On a sparse project, /relevant will hand you the least-bad matches with a straight face. Apply your own cutoff:

memories = [m for m in resp["memories"] if m["score"] >= 0.35]

Choose the threshold empirically against your own data, and re-check it when Venkai announces a ranking improvement — absolute score values are not guaranteed to stay put across releases.

3. Near-duplicates both ship

If your agent writes "we chose Postgres" on three consecutive runs, all three can occupy your top-5. Deduplicate before writing (Memory), or collapse on the way out.

Why ranking is more than similarity

The reason Venkai does not just return nearest neighbours: similarity alone is a bad selection policy for agent memory.

A hard constraint written six months ago is more relevant to today's task than a chatty fact written this morning that happens to share vocabulary with the query. Pure similarity ranks the chatty fact first. Importance and recency exist to encode "this mattered" and "this is current" as first-class signals.

That is a policy, and it is the thing Venkai ships that a vector index does not. It is not configurable per request today — see Ranking. If your domain needs a different policy, tell us.

What we do not claim

No measurement is presented here showing that this policy beats plain similarity, or beats stuffing the whole history into the prompt. No such benchmark has been published. Evaluation explains why.

Evaluate the policy against your own workload. The scores and justifications are on every result specifically so you can.