Deterministic routing for production AI traffic.
CrispNode sits between your services and upstream inference providers — shaping traffic, failing over between regions and providers, and enforcing token budgets before a runaway job reaches your invoice.
# Production failover policy
gateway:
region_affinity: "us-east"
timeout_ms: 4500
retry_on: [429, 502, 503]
pools:
- id: primary
weight: 85
max_rpm: 12000
circuit_breaker: strict
- id: secondary
weight: 15
fallback_only: true
budgets:
- tenant: "batch-jobs"
monthly_usd: 2000
on_exceed: reject
telemetry:
export: "otlp"
payload_retention: "none"
How it works
One hop between your code and every provider you use.
01 / Core capabilities
Built for teams running multi-provider stacks.
Adaptive traffic shaping
Redistribute requests across healthy endpoints when upstream latency spikes or rate limits trigger.
Cost & token quotas
Hard spend caps per service, environment or tenant key, checked before the request leaves the gateway.
Stateless payload inspection
Redact PII patterns and validate JSON schemas in memory, without writing prompts or responses to disk.
Circuit breaking
Trip unhealthy pools out of rotation and probe them back in, so one failing upstream doesn't stall every caller.
Policy as code
Routing lives in versioned YAML you review in pull requests, validate in CI and roll back like any other config.
OTLP telemetry
Latency percentiles, token throughput and status codes, exported to Datadog, Grafana or Prometheus.
02 / Integration
Drop-in compatibility in three steps.
Register upstream keys
Store provider credentials in the encrypted vault and assign them to primary and fallback pools.
Swap the base URL
Point your HTTP client or OpenAI-compatible SDK at api.crispnode.dev/v1.
Watch the telemetry
Stream per-route metrics to the tools you already use and tune weights from real traffic.
Plans from $49 / month.
Start on Developer, move to Production when the traffic is real.