System architecture¶

Components¶

flowchart TB
  subgraph edge["Edge"]
    CADDY["Caddy<br/>TLS termination + reverse proxy"]
  end

  subgraph app["Application container"]
    FA["FastAPI (uvicorn)"]
    MW["Middleware<br/>CORS · rate limit · usage counter"]
    AUTH["auth.py<br/>API key | session JWT"]
    MAIN["main.py<br/>route handlers"]
    RET["memory_retriever.py<br/>MemoryRetriever"]
    SCORE["memory_scoring.py<br/>weights, recency, frequency"]
    DB["db.py<br/>SQLAlchemy Core"]
  end

  subgraph ext["External"]
    PG[("PostgreSQL")]
    EMB["Embedding provider<br/>hashing | sentence-transformers"]
  end

  CADDY --> FA --> MW --> AUTH --> MAIN
  MAIN --> RET --> SCORE
  RET --> EMB
  MAIN --> DB --> PG
  RET --> DB

A single Python process. No workers, no queues, no cache tier, no message bus.

Module map¶

Module	Responsibility
`api/main.py`	FastAPI app, every route handler, middleware
`api/auth.py`	Credential resolution — API key or session JWT → org identity
`api/models.py`	Pydantic request schemas; the validation boundary
`api/db.py`	SQLAlchemy Core tables and every query
`api/memory_retriever.py`	Embedding providers, `MemoryRetriever`, cosine
`api/memory_scoring.py`	`DEFAULT_WEIGHTS`, recency and frequency curves, lexical tokenizer
`sdk/client.py`	Python client over the REST API
`mcp/server.py`	stdio MCP server, HTTP client to the API
`security_graph/`	Reference security knowledge graph (experimental)

Request path¶

sequenceDiagram
  participant C as Client
  participant MW as Middleware
  participant A as auth
  participant H as Handler
  participant D as db
  participant E as Embedding

  C->>MW: HTTP request
  MW->>MW: CORS check
  MW->>MW: rate limit (production only)
  MW->>A: resolve credential
  A-->>MW: 401 if absent/invalid
  A->>H: identity {org_id, auth}
  H->>H: Pydantic validation → 422
  alt write
    H->>E: embed(content)
    H->>D: INSERT + embedding_json
  else read
    H->>D: load candidates (org-scoped)
    H->>H: re-verify org per candidate
    H->>E: embed(query)
    H->>H: score, sort, truncate
  end
  H-->>MW: response
  MW->>D: record_usage(org, kind, status)
  MW-->>C: JSON

Two design choices worth calling out¶

The org check happens twice¶

Candidates are loaded through an org-keyed query, then every candidate is re-verified to belong to the caller's organization before it can reach a result. Redundant by construction — and that is the point: a stale or misrouted project id cannot leak another tenant's memories through a single missed WHERE clause.

Usage counting lives in middleware¶

Rather than each handler recording its own usage, a single middleware reads an org_id that the auth dependency stamps onto the request. One wiring point, every route covered, nothing to forget in a new handler.

The classification is deliberately coarse:

Kind	Matches
`recall`	`GET /api/context/*/relevant`
`remember`	`POST /api/context`, `POST /api/memory`
`read`	any other `GET`
`admin`	everything else

Unauthenticated requests count nothing — there is no org to attribute them to.

Deployment topology¶

flowchart LR
  U["Internet"] --> CF["Cloudflare<br/>DNS + TLS"]
  CF --> C["Caddy<br/>:80 / :443"]
  C -->|"api.venkai.fr"| API["venkai-api<br/>:8100, not host-exposed"]
  C -->|"app.venkai.fr"| DASH["static dashboard"]
  API --> PG[("PostgreSQL")]
  API --> V["/data volume"]

Port 8100 is not published to the host. All traffic goes through Caddy, so the rate limiter and the logs see real client IPs rather than a spoofable X-Forwarded-For from a directly-reachable port.

Scaling characteristics¶

Dimension	Behaviour
Requests	Stateless — horizontally scalable behind a load balancer
Memories per project	`O(n)` per query. The real ceiling.
Projects	Independent; each query touches one
Organizations	Row-level scoping; no per-tenant infrastructure
Write throughput	One synchronous embedding per write

The scaling limit is memories per project, not requests. pgvector plus an ANN index is the planned answer; see Storage.

Absent by design¶

Not present	Consequence
Cache layer	Every read hits the database
Background workers	Embedding is inline; no queue to lose data in
Read replicas	Reads and writes share one connection pool
ANN index	Linear scan per query
Multi-region	Single deployment
Per-memory delete	Supersede via `PATCH`, or delete the org

Each of these is a real limit rather than an oversight. They are listed so you can judge whether they matter for your workload before you find out.

Storage · Embeddings
Retrieval pipeline · Context pipeline