Skip to content
browse the docs

LLM API Proxy Quickstart (90 seconds)

vanzik v2.0updated view on github

You'll go from "OpenAI SDK calls api.openai.com directly" to "OpenAI SDK calls flow through Vanzik, with every tool call policy-gated" in under 90 seconds.

By the end, every chat.completions.create(...) call your agent makes passes through Vanzik, is evaluated against a YAML policy, and shows up live on a dashboard you can approve / deny from.

Prerequisites#

  • Python 3.10+ with the OpenAI SDK: pip install "openai>=1.0"
  • A valid OPENAI_API_KEY (the proxy forwards it verbatim to api.openai.com)
  • Go 1.26+ for go install: https://go.dev/dl/

This walkthrough uses the OpenAI SDK. The same pattern works for Anthropic, LangChain, and CrewAI — see examples/ for ready-to-run scripts.

30 seconds — install Vanzik#

go install lictorate.com/vanzik/v2/cmd/vanzik@latest
go install lictorate.com/vanzik/v2/cmd/vanzik-llm-proxy@latest

Confirm both binaries are on PATH:

which vanzik vanzik-llm-proxy
# /Users/you/go/bin/vanzik
# /Users/you/go/bin/vanzik-llm-proxy

If which finds nothing, add $(go env GOPATH)/bin to your shell's PATH and re-source the rc file.

30 seconds — start the policy server + proxy#

You need two Vanzik processes running side by side: the central policy server (port 8080) and the LLM API Proxy (port 8081). Generate a shared API key, then start both.

In the first terminal:

export VANZIK_API_KEY="$(openssl rand -hex 32)"

git clone https://github.com/lictorate/vanzik.git
cd vanzik

vanzik server \
    --policy configs/default.yaml \
    --dashboard \
    --watch \
    --api-key "$VANZIK_API_KEY"

Open the dashboard at http://127.0.0.1:8080/dashboard and log in with the API key. Leave this terminal running.

In a second terminal (same env, same key):

export VANZIK_API_KEY="<paste the key from the first terminal>"

vanzik-llm-proxy \
    --listen 127.0.0.1:8081 \
    --policy configs/default.yaml \
    --guard-url http://127.0.0.1:8080 \
    --api-key "$VANZIK_API_KEY"

The proxy refuses to bind to non-loopback hosts without --proxy-api-key set — that includes an empty host such as :8081, which listens on every interface — so the localhost defaults are safe by construction.

30 seconds — point your code at the proxy#

In a third terminal:

export OPENAI_API_KEY=sk-...                       # your real OpenAI key
export OPENAI_BASE_URL=http://127.0.0.1:8081/v1    # the proxy

python examples/openai-sdk-config.py

The example asks the model to list /tmp via a bash tool. The default policy's shell rules ALLOW it; the dashboard logs an ALLOW event with transport=llm_api_proxy within a second of the script firing.

Verify (the satisfying part)#

Watch the dashboard at http://127.0.0.1:8080/dashboard. You should see one ALLOW entry with:

  • transport chip: llm_api_proxy (purple)
  • scope: shell (the bundled scope map sends the bash tool to shell)
  • command: the shell command the model asked to run, e.g. ls /tmp — for shell-mapped tools the proxy checks the tool's command argument, not the tool name. The tool name (bash) is in the audit entry's request.meta.tool_name.

Now edit configs/default.yaml — add a deny rule:

rules:
  - scope: shell
    deny:
      - pattern: "rm -rf *"
    # ... existing rules ...

vanzik server reloads the policy file automatically (no flag needed; --watch only logs reload activity), and the verdict comes from the server. The proxy reloads its own copy too, but only to refresh its tool_scope_map. Modify the prompt in examples/openai-sdk-config.py to ask the model to rm -rf /etc and re-run. The model emits a tool call; Vanzik intercepts it mid-stream and rewrites the response as a synthetic assistant text:

Vanzik denied this action.

Reason: Action denied by shell deny rule
Rule: deny:shell:rm -rf *

The dashboard logs a DENY. No SDK changes, no agent code changes — the model's tool call literally never reaches your script.

That's the loop. Every tool call → policy check → audit log → live dashboard. Tool deltas in ALLOW are byte-identical to the upstream bytes; deltas in DENY/REQUIRE_APPROVAL are rewritten in place as provider-shaped synthetic refusals.

Next steps#