INFERENCE ROUTING/OPENAI-COMPATIBLE GATEWAY

Deterministic routing for production AI traffic.

CrispNode sits between your services and upstream inference providers — shaping traffic, failing over between regions and providers, and enforcing token budgets before a runaway job reaches your invoice.

Read the docs →
1 line
Swap your base URL
YAML
Versioned routing policy
No storage
Payloads stay in memory
routing-policy.yamlVALID
# Production failover policy
gateway:
  region_affinity: "us-east"
  timeout_ms: 4500
  retry_on: [429, 502, 503]

pools:
  - id: primary
    weight: 85
    max_rpm: 12000
    circuit_breaker: strict
  - id: secondary
    weight: 15
    fallback_only: true

budgets:
  - tenant: "batch-jobs"
    monthly_usd: 2000
    on_exceed: reject

telemetry:
  export: "otlp"
  payload_retention: "none"
Policies apply on deploy — no gateway restart.

How it works

One hop between your code and every provider you use.

api-servicebatch-workersupport-bot Provider A · region 1Provider A · region 2Provider B CrispNode gateway › policy match› budget check› weighted route› retry / fail over fallback route

01 / Core capabilities

Built for teams running multi-provider stacks.

MOD-01

Adaptive traffic shaping

Redistribute requests across healthy endpoints when upstream latency spikes or rate limits trigger.

MOD-02

Cost & token quotas

Hard spend caps per service, environment or tenant key, checked before the request leaves the gateway.

MOD-03

Stateless payload inspection

Redact PII patterns and validate JSON schemas in memory, without writing prompts or responses to disk.

MOD-04

Circuit breaking

Trip unhealthy pools out of rotation and probe them back in, so one failing upstream doesn't stall every caller.

MOD-05

Policy as code

Routing lives in versioned YAML you review in pull requests, validate in CI and roll back like any other config.

MOD-06

OTLP telemetry

Latency percentiles, token throughput and status codes, exported to Datadog, Grafana or Prometheus.

Product overview →

02 / Integration

Drop-in compatibility in three steps.

No SDK lock-in
STEP 01

Register upstream keys

Store provider credentials in the encrypted vault and assign them to primary and fallback pools.

STEP 02

Swap the base URL

Point your HTTP client or OpenAI-compatible SDK at api.crispnode.dev/v1.

STEP 03

Watch the telemetry

Stream per-route metrics to the tools you already use and tune weights from real traffic.

Plans from $49 / month.

Start on Developer, move to Production when the traffic is real.

See pricing