Proxy Architecture

vanzik v2.0updated 2026-10-21view on github

This doc covers the cross-cutting decisions that apply to both Vanzik proxies. The two follow-up docs cover the proxy-specific wire format and data plane:

docs/MCP_GATEWAY.md — vanzik-mcp-gateway
docs/LLM_API_PROXY.md — vanzik-llm-proxy

Read this one first.

1. The hero claim, restated honestly#

Vanzik exposes its firewall as a wire-level enforcement boundary via two integration paths:

MCP Gateway — a JSON-RPC bridge that sits between an MCP client (Claude Desktop, Cursor, IDE plugins) and one or more downstream MCP servers. Every tools/call is gated by the policy engine before reaching the upstream.
LLM API Proxy — an HTTP server that speaks /v1/chat/completions (OpenAI) and /v1/messages (Anthropic) wire formats and forwards to the real upstream after gating any tool calls the model emits — including tool calls that arrive inside an SSE stream.

The Python and TypeScript SDKs are the compatibility tier for direct callers: code paths that talk to Vanzik via Guard.check remain a first-class integration, but they are not the only way to get coverage.

1.1 What "wire-level" does and does not mean#

The two proxies are wire-level for the configured base URL or the gateway subprocess. An agent can still bypass either path by:

ignoring OPENAI_BASE_URL / ANTHROPIC_BASE_URL and dialing the real upstream directly,
launching an MCP server with no Vanzik gateway in front of it,
using a tool the SDK doesn't know about.

Vanzik is a defensive guardrail for cooperating agents, not a mandatory syscall interceptor (README, "Limitations & Threat Model").

2. Shared substrate#

The two proxies do not duplicate the policy engine, audit logger, approval queue, or SSE bus. They both consume primitives provided by the central server packages.

2.1 Policy: every decision comes from the central server#

Neither proxy holds a *policy.Engine. Every decision is an HTTP POST to the central server's /v1/t/<tenant>/check, where <tenant> is the proxy's --tenant-id (default local), made through the shared client in pkg/internal/gateclient (§ 2.6). The central server runs Engine.Check against its own copy of the policy.

Each proxy also loads the policy YAML given by --policy through a policy.FilePolicyProvider, but only to read tool_scope_map (which scope a tool name maps to). The provider's Watch swaps that snapshot when the file changes. The verdict still comes from the central server's policy, so point both at the same file. The MCP gateway requires --policy in its default strict mode; the LLM API proxy falls back to its bundled tool map without it.

How many checks one call makes:

LLM API proxy: one /v1/check per tool call in the model's response, against the tool's mapped scope.
MCP gateway, --policy-mode strict (default): an mcp_tool check, then, if it allows and the tool maps to a scope, a check against the mapped scope (one per path when the arguments carry a paths array). A mapped call therefore makes two checks. --policy-mode fast makes only the mcp_tool check. See MCP_GATEWAY.md § 4.4.

2.2 Audit: written by the central server#

Neither proxy writes an audit log. The central server audits each /v1/check it answers, like any other check; the proxy stamps meta.transport on the request so the entry carries the right transport (§ 3). An MCP gateway call in strict mode that maps to a scope therefore leaves two entries, one per check.

Two exceptions write elsewhere:

--fail-mode fail-closed-with-audit: while the central server is unreachable, each fail-closed denial is appended to the local --fail-audit-log JSONL file, in the audit.Entry shape (both proxies).
When the LLM API proxy refuses a malformed tool call itself, it posts the DENY it sent to the client to the central server's POST /v1/t/<tenant>/audit, which appends it as is.

2.3 Approval queue and SSE bus via `pkg/proxy.ApprovalQueue`#

The existing Vanzik HTTP server (vanzik server) owns the approval queue and the SSE bus. The two proxies do not maintain their own approval state. When a check returns REQUIRE_APPROVAL the proxy:

receives ApprovalID and ApprovalURL in the CheckResult,
surfaces them to the agent (synthetic refusal for the LLM proxy; isError: true content block for the MCP gateway),
the operator approves on the dashboard (or via vanzik approve CLI) — the resolution lands on the central guard server's ApprovalQueue and is broadcast on its SSE bus.

The dashboard's mixed feed therefore shows MCP-gateway approvals, LLM-proxy approvals, and direct-SDK approvals on the same wire — chipped by the new transport field so operators can filter at a glance.

2.4 Notifier via `pkg/notify.Dispatcher`#

Webhook / Slack / console notifications are the central guard server's job. The proxies do not run their own dispatcher — they receive REQUIRE_APPROVAL / DENY CheckResults back from /v1/check and the central server has already fanned out the event by the time the proxy sees the response.

2.5 Diagram — single host#

                        ┌─────────────────────────────────┐
                        │      vanzik server           │
                        │  (central host: 127.0.0.1:8080) │
   /v1/check ─────►     │  • Engine.Check                 │
   /v1/approve ─────►   │  • ApprovalQueue + SSE bus      │
   /v1/audit ─────►     │  • BufferedAsyncLogger          │
                        │  • Notifier (webhook/Slack)     │
                        │  • Dashboard (/dashboard, /api) │
                        └─────────────────▲───────────────┘
                                          │
              ┌───────────────────────────┼───────────────────────────┐
              │                           │                           │
        /v1/check                   /v1/check                   /v1/check
              │                           │                           │
   ┌──────────┴──────────┐    ┌───────────┴────────────┐    ┌─────────┴─────────┐
   │ vanzik-         │    │ vanzik-llm-proxy   │    │ Python/TS SDKs    │
   │ mcp-gateway         │    │ (127.0.0.1:8081)       │    │ (Guard.check)     │
   │ (stdio child of     │    │ • /v1/chat/completions │    │                   │
   │  Claude Desktop)    │    │ • /v1/messages         │    │                   │
   │ • bridges JSON-RPC  │    │ • streaming pause/     │    │                   │
   │ • spawns N upstream │    │   resume/rewrite       │    │                   │
   │   MCP servers       │    │                        │    │                   │
   └─────────────────────┘    └────────────────────────┘    └───────────────────┘
       ▲       │                       ▲      │                     ▲
       │       ▼                       │      ▼                     │
   stdio in   stdio out             OpenAI/Anthropic         direct agent code
   (client)   (downstream            (real upstream)           (LangChain, etc.)
              MCP servers)

2.6 Gate client via `pkg/internal/gateclient`#

The /v1/check client (tenant-scoped URL, schema_version, bearer auth, 64 KiB response cap), the fail-mode translation and the shared flags (§ 7) are implemented once in pkg/internal/gateclient and used by both proxies; each keeps only its own scope mapping and argument projection. The package is internal, not public API; each proxy's synthetic rule strings remain its own stable contract.

3. Audit transport tag#

Every audit entry is chipped by how the agent talked to the firewall. Three values for the proxies and the SDKs, and one per coding-agent hook (CODING_AGENTS.md):

transport	source
`mcp_gateway`	`vanzik-mcp-gateway`
`llm_api_proxy`	`vanzik-llm-proxy`
`sdk`	direct `/v1/check` callers (Python/TS SDKs, hand-rolled HTTP clients, framework adapters)
`claude_code`	`vanzik hook claude-code`
`cursor`	`vanzik hook cursor`
`gemini_cli`	`vanzik hook gemini-cli`
`codex_cli`	`vanzik hook codex`

3.1 Where the field lives#

Decision: top-level Entry.Transport string field.

Alternatives considered:

Meta["transport"] string — nests under existing free-form bag. Rejected because the dashboard chip and the Logger.Query filter want a stable indexed field, not a string lookup inside a map. The performance gap on a JSON-Lines log is small but the API gap on Query(filter QueryFilter{Transport: "mcp_gateway"}) is glaring.
Bump audit schema_version from 2 to 3. Rejected because the field is purely additive: existing readers ignore unknown top-level keys; existing writers continue to emit v2 entries with no transport field, which readers treat as "sdk".

3.2 Reader rules#

New entries MUST include "transport".
Entries with no "transport" key default to "sdk" at read time — the SDK path was the only writer before the proxies existed.
Logger.Query accepts Transport string in the filter; matches by exact equality. Empty filter value matches all.

3.3 Wire schema impact#

pkg/proxy/schema/v1/schema.json — the cross-language contract that governs /v1/check request and response — is not affected. The transport tag is an audit-side concern. The proxy that calls /v1/check knows its own transport and stamps it on the request as meta.transport; the central server reads it from there when it writes the entry. Nothing is negotiated.

The CheckResult schema stays at schema_version: "v1". No bump.

4. Binary structure#

4.1 Decision: two binaries#

The proxies ship as two binaries alongside the central vanzik binary:

binary	role
`vanzik`	central server (`server`, `validate`, `audit`, etc.)
`vanzik-mcp-gateway`	stdio bridge, child of an MCP host (Claude Desktop)
`vanzik-llm-proxy`	HTTP server, sidecar for OpenAI/Anthropic-compatible callers

All three live under cmd/:

cmd/vanzik/main.go
cmd/vanzik-mcp-gateway/main.go
cmd/vanzik-llm-proxy/main.go

Each cmd/vanzik-*/main.go is a thin entry point. Real logic lives in pkg/mcpgw/ and pkg/llmproxy/. Neither imports pkg/proxy: the server, approval queue and audit pipeline stay in the central binary.

4.2 Why not subcommands of `vanzik`?#

Settled decision: three separate binaries. The MCP gateway must be one — MCP clients spawn it via claude_desktop_config.json's command field with stdin/stdout reserved for JSON-RPC, which cannot coexist with a long-running HTTP daemon in the same process. The LLM proxy is separate for failure isolation and so sidecar deployments can ship proxy fixes without redeploying the central server. Versions stay synced via bump-version.sh (§ 4.3).

4.3 Versioning#

All three binaries share a single release version. bump-version.sh sed-replaces the version string across cmd/vanzik/main.go, cmd/vanzik-mcp-gateway/main.go, cmd/vanzik-llm-proxy/main.go, and the SDK manifests.

vanzik-mcp-gateway --version and vanzik-llm-proxy --version print the binary's name, version and commit in the same form as vanzik version, e.g. vanzik-mcp-gateway 2.0.0 (0d2852e). The wire binaries take the --version flag, not a version subcommand.

4.4 Same `pkg/` codebase#

Critical: there is one policy engine, one audit format, one approval queue type. The two proxy binaries import the same packages and configure them differently. Implementation drift between the proxies fails CI because the pkg/ tests run for all callers.

5. Deployment topologies#

5.1 Local-only (developer machine)#

All three binaries on 127.0.0.1. Everything chats over loopback.

vanzik server --port 8080 --dashboard --policy configs/default.yaml
vanzik-mcp-gateway --guard-url http://127.0.0.1:8080 --policy configs/default.yaml --upstream "fs:npx -y @modelcontextprotocol/server-filesystem /tmp"
vanzik-llm-proxy --listen 127.0.0.1:8081 --guard-url http://127.0.0.1:8080

The agent's environment:

OPENAI_BASE_URL=http://127.0.0.1:8081/v1
ANTHROPIC_BASE_URL=http://127.0.0.1:8081

Note the asymmetry: the OpenAI SDK appends paths under OPENAI_BASE_URL including the /v1 segment that the proxy registers (POST /v1/chat/completions), so the env var must include /v1. The Anthropic SDK convention is the opposite — ANTHROPIC_BASE_URL is the origin and the SDK appends /v1/messages itself, so the env var must NOT include a /v1 suffix.

…and Claude Desktop's claude_desktop_config.json points at the gateway binary.

5.2 Single-host server#

One host runs vanzik server. Gateways and proxies run as sidecars on the same host or on agent-facing hosts pointing at the central server's /v1/check over the network. Set --api-key on the central server and pass it to every gateway/proxy via --api-key.

5.3 Multi-tenant (shipped in v0.6)#

A single central server serves many tenants:

per-tenant policies live in the durable store, managed via vanzik tenant put/list/rm (see CLI.md),
requests reach a tenant through /v1/t/{tenant}/…,
the approval queue, rate limiter, cost accounting, and audit query are all sharded by tenant ID,
the proxies stamp their tenant via --tenant-id (default local).

5.4 Multi-node (shipped in v1.0, PostgreSQL)#

Multiple central-server replicas can share state — tenant policies, approvals, rate-limit and cost consumption — by pointing --store-dsn at one PostgreSQL and giving each replica a --node-id. Each node's memory stays authoritative for its own /v1/check decisions (the hot path never performs a synchronous database call); a background reconciler merges the other nodes' state in every --reconcile-interval (default 2s). As a consequence, distributed rate/cost limiting is bounded-overshoot rather than globally strict, and cross-node approval visibility lags by at most one interval. See OPERATIONS.md and COMPATIBILITY.md.

6. Failure modes#

6.1 Central guard server (`vanzik server`) is unreachable#

This is the case the SDK already handles (fail-mode: deny|allow). Both proxies adopt the same flag for parity:

--fail-mode deny|allow|fail-closed-with-audit   (default: deny)

value	behavior on `/v1/check` failure
`deny`	synthesise `DENY` with `Rule="deny:<gateway>:fail_closed"` (LLM proxy emits `deny:llm_api_proxy:fail_closed`; MCP gateway emits `deny:gateway:fail_closed`). The agent sees a deny. Default.
`allow`	synthesise `ALLOW` with `Rule="allow:<gateway>:fail_open"`. Use only in trusted dev environments. An explicit opt-in: neither proxy logs the mode at startup.
`fail-closed-with-audit`	synthesise `DENY` with a distinct `Rule="deny:<gateway>:fail_closed_audit"` so dashboards can break out central-server-outage events from plain fail-closed denials, and append the denial to a local fallback audit file (`--fail-audit-log`, default `vanzik-fail-audit.jsonl`, canonical `audit.Entry` JSONL) so the outage window stays reconstructable without the central server.

Independent of --fail-mode: when /v1/check answers 2xx but with an unrecognised decision string, both proxies deny with the stable Rule="deny:<gateway>:invalid_response" (deny:llm_api_proxy: invalid_response / deny:gateway:invalid_response) so a misbehaving or version-skewed central server is visible on dashboards instead of silently passing through.

The SDKs offer the same choice: Python's fail_mode constructor argument and TypeScript's failMode ("deny", the default, or "allow", an explicit opt-out for callers whose threat model treats the firewall as best-effort).

6.2 MCP gateway: downstream subprocess crashes#

Reconnect with capped backoff (1s → 2s → 5s → 30s → 60s, cap 60s). The namespace is marked degraded; tools/list excludes its tools while the flag is set; tools/call returns the JSON-RPC error -32002 (tools/call: upstream "<ns>" unavailable, data {"namespace": "<ns>"}). Detail in MCP_GATEWAY.md § 7.

6.3 LLM API proxy: upstream OpenAI/Anthropic returns 5xx or times out#

Pass-through: a 5xx is forwarded with the upstream's status code and body verbatim. Errors are not transformed into Vanzik-shaped JSON — the agent's own retry/error-handling path gets to see the real upstream response. When the upstream can't be reached or times out, the proxy answers 502 with an upstream error JSON body.

No audit entry is written in either case. The proxy audits only through the /v1/check calls it makes for tool calls (§ 2.2), and a failed upstream response carries no tool calls to check.

6.4 Proxy itself crashes#

Both proxies are stateless w.r.t. policy decisions. Restart loses in-flight gating context only. The MCP gateway's degraded-namespace flags are reset; the LLM proxy's per-stream buffers are dropped (the client's stream errors out and the agent retries).

In-flight REQUIRE_APPROVAL records live on the central server's ApprovalQueue, not in the proxy. Restarting a proxy does not lose approvals.

7. Configuration parity#

Both binaries share these flags:

flag	meaning	default
`--guard-url`	central server `/v1/check` base URL (from `VANZIK_URL` env if unset)	`http://127.0.0.1:8080`
`--api-key`	bearer token for `/v1/check` (from `VANZIK_API_KEY` env if unset, then the key `vanzik setup` saved)	unset (warn)
`--tenant-id`	tenant header value	`local`
`--fail-mode`	`deny` / `allow` / `fail-closed-with-audit`	`deny`
`--log-level`	`info` / `debug`	`info`
`--version`	print `<name> <version> (<commit>)` and exit	—

Proxy-specific flags (upstreams, listen address, etc.) live in the respective doc.

8. Health & observability#

The two proxies have very different surfaces here:

LLM API Proxy registers /healthz (note: not /health) returning a flat status object:
```
{
  "status": "ok",
  "version": "2.0.0",
  "transport": "llm_api_proxy",
  "uptime_s": 412
}
```
No guard_reachable field, no warnings array. The proxy does not expose its own /metrics endpoint; per-request metrics flow through the central server's /v1/check instrumentation (see § 8.1 below).
MCP Gateway is a stdio JSON-RPC binary with no HTTP surface at all — no /health, no /metrics. Operators inspect it via the process's stderr log lines. Health is implicit in whether the host client (Claude Desktop, Cursor, etc.) reports the gateway as alive.

8.1 Where the metrics actually live#

Per-request gating metrics are emitted on the central server's /metrics endpoint, distinguished by the Entry.Transport audit field:

metric	source
`vanzik_checks_total`	central server
`vanzik_request_duration_ms`	central server
`vanzik_llmproxy_buffer_overflow_total`	LLM API Proxy (process-local)
`vanzik_llmproxy_streams_active`	LLM API Proxy (process-local)

The vanzik_llmproxy_* series are counted in the proxy's own process, which serves no /metrics endpoint. Because pkg/metrics is one shared registry, the central server's /metrics does list all six series, but always at 0: nothing in the server process increments them. Don't alert on them. Observable proxy behaviour (denials, overflows surfacing as denials) lands in the central server's audit stream and its other metrics.

The MCP gateway has no scrape surface; its decisions surface in the central server's audit stream via Entry.Transport == "mcp_gateway".

9. Notable cross-cutting decisions#

A few cross-cutting design decisions are worth surfacing in one place; the proxy-specific docs cover the full detail.

MCP scope dual-check (mcp_tool + mapped scope) is enabled by default behind a --policy-mode strict|fast flag. Dual-check honours existing scope rules without operators having to duplicate them as mcp_tool rules. See MCP_GATEWAY.md § 4.4.
LLM proxy buffer bound on tool-call accumulation is 1 MiB per content block (--max-buffer-bytes 1048576). Anything larger triggers a synthetic refusal. See LLM_API_PROXY.md § 5.6.
Approval re-submission for the MCP gateway. When a tool call returns REQUIRE_APPROVAL and the operator approves, the client retries with _meta.dev.vanzik/approval_id echoed in the call params; the gateway looks up the resolution and either forwards (ALLOW) or refuses (DENY). The _meta namespace prefix dev.vanzik/ is reserved per the MCP spec's _meta rules. Detail in MCP_GATEWAY.md § 6.

10. References (external)#

MCP spec — https://modelcontextprotocol.io/specification/ (gateway targets 2025-11-25; the Python SDK adapter pins the older 2024-11-05).
Anthropic Messages API — https://platform.claude.com/docs/en/api/messages.
OpenAI Chat Completions API — https://platform.openai.com/docs/api-reference/chat/create (streaming wire shape cross-checked in LLM_API_PROXY.md § 5.1).
Prior art consulted (architectural reference only): sparfenyuk/mcp-proxy, MCP Inspector, LiteLLM, mitmproxy, go-mitmproxy.