The model control plane for AI apps

Make every model call observable, controllable, Built to recover.

Keep an OpenAI-compatible interface and your existing identity system. NeoRouter handles app isolation, routing, cost settlement, risk controls, and safe experiments on the request path.

Identity and app isolationRisk and operations loopSafe experiments and rollback

Start integrating Developer guide

Keep your user system. Keep provider logic out of your app.

quickstart.js

// Keep your SDK. Move control to NeoRouter.
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.neorouter.app/v1",
  apiKey: process.env.NEOROUTER_API_KEY
});

const response = await client.chat.completions.create({
  model: "openai/gpt-5",
  messages: [{ role: "user", content: "Hello" }]
});

SWITCH MODEL OR POLICY

POST/v1/chat/completions 482 ms200 OK

AUTH → ROUTE → SETTLE → OBSERVE → CONTROL CONTROL PLANE · GLOBAL

One control plane for today's language models and tomorrow's multimodal workloads

LANGUAGE LIVEGPTClaudeGeminiQwenMiniMax

VOICE ROADMAPSpeechTranscription

IMAGE ROADMAPGenerateEdit

VIDEO ROADMAPGenerateTransform

REQUEST GOVERNANCE

One call, controlled from identity to outcome.

NeoRouter does more than proxy traffic. It verifies callers, applies policy, reserves cost, selects a route, settles usage, and turns anomalies into operational work.

Your app sends a requestapi.neorouter.appA reliable, explainable outcome

01IDENTITY

Verify identity

Bring your identity provider or a bounded API key.

02ISOLATION

Isolate the app

Bind app, subject, permissions, and capabilities.

03BUDGET

Reserve cost

Check credit, concurrency, and worst-case cost first.

04ROUTING

Choose a route

Balance quality, latency, price, and live health.

05SETTLEMENT

Settle exactly

Account for actual input, cache, and output usage.

06OPERATIONS

Observe and control

Connect requests, risk, action, notification, and recovery.

ONE CONTROL PLANE

Turn fragmented model work into one operating loop.

Development, platform, security, and operations teams share the same request truth while controlling the layer each team owns.

INTEGRATE

One API contract

Provider SDKs, protocols, streaming, and error formats all differ.

Use one interface, logical models, and presets while expanding to more modalities over time.

OpenAI-compatible · SSE · Presets

AUTHENTICATE

Bring your identity

A shared key cannot express app, user, scope, environment, and budget boundaries.

Normalize your identity provider into app and subject context; use short-lived, scoped, capped keys for automation.

JWT/JWKS · Scopes · Bounded keys

STAY RELIABLE

Keep requests moving

Upstream limits, route incidents, and model removals become visible user errors.

Retry only safe failures, fall back across routes and models, and stop propagation with circuit breakers.

Retry · Fallback · Circuit breaking

OPERATE

Explain every call

Fragmented logs cannot explain where latency, errors, cost, and fallback came from.

Connect requests, attempts, exact usage, queues, settlement, and notifications in one operational view.

Requests · Attempts · Exact ledger

CONTROL RISK

Turn signals into action

Leaks, abuse, and usage spikes can multiply cost before a team notices.

Move from detection and shadow validation to owned response, temporary controls, recovery, and audit.

Detect · Control · Audit · Recover

EXPERIMENT

Release AI changes safely

Small prompt, model, and parameter changes can shift quality, cost, and latency together.

Bucket versions consistently, combine offline evals with online outcomes, then promote or roll back behind guardrails.

A/B/n · Evals · Guardrails · Rollback

RISK & OPERATIONS

When something changes, know what to do next.

Bring platform incidents, subject anomalies, and billing risk into one queue with ownership, workflow state, related requests, and a complete audit trail.

Open incidents07Across platform and subject signals

Monitoring cycle15 minWith baselines and recovery checks

Rule modeShadowValidate before promoting to Active

RISK SIGNALSCOPEOWNERSTATE

Abnormal token growthRISK-2047app · subject_8F2AOn-callReview

Recurring provider errorsRISK-2039provider · routePlatformWatching

Test key approaching its capRISK-2028api_key · budgetSecurityResolved

CONTROLLED SUBJECTsubject_8F2Aapp.supermuse · production

Controls expire automatically in 46 minutes

01Lower concurrency

Protect the platform while preserving a slower usable path.

Active

02Cap total credit

Reserve atomically and settle against actual call cost.

Active

03Pause generation

Pause the risky capability without blocking the rest of the app.

Standby

Every control is app- and subject-bound, time-limited, and written to the audit trail.

Request healthHealthy

Watch latency, status, and fallback behavior continuously.

p95 · 684 ms

Usage queueOn time

Batched events are processing inside the target window.

lag · 1.8 s

Cost settlementSettled

Reserved credit and actual usage are reconciled.

pending · 0

Notification deliveryDelivered

Webhooks include retries, expiry, and daily digests.

retrying · 0

IDENTITY & DATA BOUNDARIES

Bring your system. Do not reduce trust to a shared key.

NeoRouter normalizes identity provider, app, subject, scope, and budget. Storage can evolve while the product boundary stays stable.

01 · AUTH

Bring your identity

Verify JWT/JWKS or an identity adapter and normalize external identity into app, subject, and scope context.

Identity provider · App · Subject

02 · KEYS

Bounded, not permanent

Issue expiring, scoped, credit-capped API keys for development, automation, and controlled testing, then revoke anytime.

Expiry · Scopes · Credit cap

03 · SECRETS

Credentials stay invisible

Rotate provider credentials through write-only slots; never echo plaintext into the console, logs, or business database.

Write-only · Rotation · Audit

DATA MINIMIZATION BY DEFAULT

Store only the metadata operations needs by default.

Prompts and responses pass through the proxy but are not written to durable storage by default. Content evaluation requires explicit app-level sampling, retention, and deletion policy.

App-bound pseudonymous subject references
Bounded diagnostics with sensitive-data redaction
Role-based administration with a complete audit trail

ROUTING & RELIABILITY

Define the outcome. Let the control plane choose the route.

Before each request leaves the gateway, NeoRouter weighs price, quality, latency, live health, and app policy. Failures are retried only inside safe boundaries.

ROUTER · LIVEpolicy/default-v2···

01 · APP REQUESTGenerate response

modality: text
region: auto

active policyNEOROUTER

GPT-5Live

412 ms

Best live latency in this region

SELECTED ROUTE

Claude SonnetLive

92 eval

Highest evaluation score for this task

Qwen MaxLive

$0.8 / 1M

Lowest projected call cost

Gemini 2.5 ProLive

99.99 %

Healthiest independent fallback route

Models, routes, and providers fail at their own scope, so one incident does not become a global outage.policy: fastest · decision: 8ms

EXPERIMENTS & EVALUATION

Turn prompt, model, and policy changes into verifiable releases.

Online experiments measure real business outcomes. Offline evals test output quality. Both share versions, cost, latency, risk guardrails, and release decisions.

ONLINE EXPERIMENTEXP-0142

Support answer policy · Prompt v4

Bucket pseudonymous subjects consistently, compare prompt versions in the same cohort, and keep models and parameters attributable.

Baselineprompt/support-v380%

Candidateprompt/support-v420%

Candidate traffic allocation

Automatic protection is on

Pause the candidate if error, cost, or latency crosses its limit.

Healthy

EXPERIMENT RESULTSQuality, cost, and reliability together

Sample data

Business quality+6.8%vs baseline

Cost per call−12%vs baseline

P95 latency612 ms−31 ms

Format compliance99.4%guardrail passed

Current confidence87%

The promotion threshold is not met yet. Keep collecting samples instead of declaring a winner too early.

Unlocks after sample and guardrail conditions pass

Offline replay

Compare prompts, models, and parameters on curated and historical cases before a visible regression reaches production.

Datasets · Regression · Pairwise

Online outcomes

Attribute version exposure to completion, adoption, rating, or conversion events returned by the app.

Stable buckets · Outcome API

Safe promotion

Gate quality, cost, errors, and latency together, then ramp progressively or roll back immediately.

Guardrails · Promote · Rollback

Experiments record only version, bucket, and operational metrics by default. Content sampling and LLM grading require an explicit app data policy.

DEVELOPER EXPERIENCE

One interface for your existing app and identity system.

Keep OpenAI SDKs, URLSession, or standard HTTP. Production apps can carry an existing user session; backends and automation can use bounded API keys.

✓Decouple providers with logical models and presets
✓Consistent JSON, errors, and SSE streaming
✓Identity adapters, app isolation, and request diagnostics

Developer guide →

import OpenAI from "openai";

const neo = new OpenAI({
  baseURL: "https://api.neorouter.app/v1",
  apiKey: process.env.NEOROUTER_API_KEY
});

const result = await neo.chat.completions.create({
  model: "@preset/default",
  messages: [{ role: "user", content: "Design a quiet onboarding flow." }]
});

import OpenAI

let config = OpenAI.Configuration(
  token: ProcessInfo.processInfo.environment["NEOROUTER_API_KEY"]!,
  host: "api.neorouter.app"
)

let neo = OpenAI(configuration: config)
let result = try await neo.chats(query: .init(
  messages: [.user(.init(content: .string("Hello")))],
  model: "openai/gpt-5"
))

from openai import OpenAI

neo = OpenAI(
  base_url="https://api.neorouter.app/v1",
  api_key=os.environ["NEOROUTER_API_KEY"]
)

result = neo.chat.completions.create(
  model="@preset/default",
  messages=[{"role": "user", "content": "Summarize this document."}]
)

curl https://api.neorouter.app/v1/chat/completions \
  -H "Authorization: Bearer $NEOROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "@preset/default",
    "messages": [{"role":"user","content":"Write a launch note."}]
  }'

api.neorouter.appOpenAI compatible

TRANSPARENT COST

Know where every call spends, and stop runaway cost early.

Model rates, cached tokens, output tokens, platform rules, and actual settlement remain attributable, with boundaries per app, subject, or test key.

USAGE BASED01

PAYGtransparent model rates and platform rules

✓Exact request-level usage ledger
✓Credit reservation before and settlement after calls
✓Cost boundaries by app, subject, and key
✓Alerts and temporary controls for anomalies

Explore billing↗

Prices, versions, and actual usage remain attributable in the control plane.

SHIP AI CAPABILITY, NOT OPERATIONAL DRAG

Connect models safely. Operate them continuously. Evolve them with confidence.

Put identity, routing, cost, risk, and experiments in one control plane from the first request.

Start integrating↗Developer guide→