Kubernetes (Helm)

Prerequisites

• kubectl configured against your cluster
• Helm ≥ 3.8
• A cluster with default StorageClass that supports ReadWriteOnce PVCs

1. Install

The chart is published to GHCR as an OCI artifact — no helm repo add needed. Minimal install with OpenAI key and an Ingress:

helm install relay oci://ghcr.io/geeper-io/charts/relay \
  --version <version> \
  --set secrets.openaiApiKey=$OPENAI_API_KEY \
  --set ingress.enabled=true \
  --set ingress.hosts[0].host=proxy.internal \
  --set ingress.hosts[0].paths[0].path=/ \
  --set ingress.hosts[0].paths[0].pathType=Prefix

What gets created:

• Deployment — proxy pods
• Service (ClusterIP on 8000)
• Ingress
• Secret (<release>-llm-proxy) — API keys and provider credentials
• Secret (<release>-master-key) — auto-generated PROXY_MASTER_KEY
• PersistentVolumeClaim × 2 — ChromaDB data (10 Gi) and knowledge base (5 Gi)
• Bitnami PostgreSQL StatefulSet + PVC (20 Gi)

PROXY_MASTER_KEY auto-generation

On first install the chart generates a random 32-character key and stores it in a dedicated Secret. On every subsequent helm upgrade, the chart reads the existing Secret via lookup() so the key is never rotated unintentionally. The Secret has helm.sh/resource-policy: keep — helm uninstall leaves it in place so a reinstall picks up the same key.

Retrieve the master key after installation:

kubectl get secret llm-proxy-master-key -o jsonpath='{.data.PROXY_MASTER_KEY}' | base64 -d

4. Create your first API key

MASTER_KEY=$(kubectl get secret llm-proxy-master-key \
  -o jsonpath='{.data.PROXY_MASTER_KEY}' | base64 -d)

USER_ID=$(curl -s -X POST \
  'https://proxy.internal/internal/users?external_id=alice%40example.com' \
  -H "Authorization: Bearer $MASTER_KEY" | jq -r .id)

curl -X POST \
  "https://proxy.internal/internal/api-keys?user_id=$USER_ID&name=team-alpha&scopes=chat" \
  -H "Authorization: Bearer $MASTER_KEY"

Pass repeated scopes and optional expires_at query parameters as needed; see First API key.

Upgrading

helm upgrade relay oci://ghcr.io/geeper-io/charts/relay \
  --version <new-version> \
  --reuse-values

--reuse-values preserves all previously set values. The PROXY_MASTER_KEY is always preserved regardless — it’s read from the cluster Secret, not from values.

Production checklist

Item	Recommended setting
Replicas	replicaCount: 2 minimum (requires RWX storage — see Scaling)
Resources	Default requests.memory: 1500Mi covers spaCy model load
Ingress TLS	ingress.tls with cert-manager
Secrets	secrets.create: false + secrets.existingSecret from Vault/ESO
Redis	redis.enabled: true for multi-replica rate limiting and caching
Monitoring	prometheus.serviceMonitor.enabled: true
API docs/CORS	Keep docs off; configure explicit config.corsAllowedOrigins only when a browser client needs it

Common values

values-prod.yaml

replicaCount: 2

image:
  repository: ghcr.io/geeper-io/relay
  tag: "1.2.0"

secrets:
  create: false
  existingSecret: llm-proxy-secrets     # pre-provisioned by Vault/ESO
  existingMasterKeySecret: ""           # leave empty = auto-generate

redis:
  enabled: true

ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
  hosts:
    - host: proxy.internal
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: llm-proxy-tls
      hosts:
        - proxy.internal

prometheus:
  serviceMonitor:
    enabled: true

helm upgrade --install relay oci://ghcr.io/geeper-io/charts/relay --version <version> -f values-prod.yaml

See Helm values reference for the full list of options.

Evaluation workload

Enable an in-cluster retrieval gate during install or upgrade:

evaluations:
  workload: Job
  mode: retrieval
  config:
    k: 5
    minimum_recall: 1.0
  cases: |
    {"id":"auth","query":"Where is authentication implemented?","relevant_ids":["a4d2f98b72e6c941"]}

helm upgrade --install relay oci://ghcr.io/geeper-io/charts/relay \
  --version <version> --namespace relay --create-namespace \
  -f values-prod.yaml --wait --wait-for-jobs

kubectl -n relay logs -l app.kubernetes.io/component=evaluator --tail=-1

The evaluator reaches Relay through its ClusterIP Service and reports pass/fail through Job status. Set workload: CronJob and schedule for periodic drift checks. Generation evaluations require a dedicated Relay API key referenced through evaluations.apiKeySecret.