Skip to content

Architecture at a glance

A short version of the full picture. The detail lives under Architecture.

flowchart TB
  subgraph clients["Clients"]
    SDK["Python SDK"]
    MCP["MCP server<br/>(stdio, 15 tools)"]
    HTTP["Any HTTP client"]
    DASH["Dashboard<br/>app.venkai.fr"]
  end

  API["FastAPI application<br/>api.venkai.fr"]

  subgraph internals["Inside the API"]
    AUTH["Auth<br/>API key or session JWT"]
    RL["Rate limiter"]
    WRITE["Write path<br/>validate → embed → persist"]
    READ["Read path<br/>MemoryRetriever"]
  end

  STORE[("SQLAlchemy store<br/>Postgres in prod, SQLite in dev")]
  EMBED["Embedding provider<br/>hashing (default) | semantic"]

  SDK --> API
  MCP --> API
  HTTP --> API
  DASH --> API
  API --> RL --> AUTH
  AUTH --> WRITE
  AUTH --> READ
  WRITE --> EMBED
  WRITE --> STORE
  READ --> STORE
  READ --> EMBED

The two paths

Write (POST /api/memory): validate the type against the six allowed values → resolve or create the project → embed content → insert the row with its embedding. One round trip, no queue, no background job.

Read (GET /api/context/{project}/relevant): load candidates for the project (bounded by VENKAI_RETRIEVAL_MAX_CANDIDATES, default 10 000) → re-verify each belongs to the caller's organization → embed the query → score every candidate → sort → return the top limit.

Scoring is a fixed linear blend:

score = 0.40 · similarity      (cosine, query vs memory embedding)
      + 0.20 · recency         (exponential decay, 14-day half-life)
      + 0.25 · importance      (the value you supplied at write time)
      + 0.15 · frequency       (log-scaled access count)

Weights live in venkai/api/memory_scoring.py as DEFAULT_WEIGHTS. They are not configurable per request today.

Deliberate simplicity

Three things are simpler than you might expect, on purpose:

  • No ANN index. Candidates are scored in a Python loop. At the current scale that is fast enough and exactly right; pgvector is a Planned drop-in for the same cosine semantics when it stops being.
  • Embeddings are JSON text, in a contexts.embedding_json column — not a native vector type. Same reason.
  • No background workers. Embedding happens inline on the request that writes the memory. A write is slower; there is no queue to lose data in.

Read System architecture for the component-level view, and Retrieval pipeline for the step-by-step of a single query.