Braintrust vs OpenRouter: model marketplace vs measured routing

---

Quick comparison: Braintrust vs OpenRouter (2026)

Connect model routing to evaluation and release control. Start free with Braintrust.

---

What OpenRouter is and what it optimizes for

OpenRouter provides a single API key and a credit balance for calling models from multiple providers. Requests use an OpenAI-compatible schema, and models can be selected without establishing a separate billing relationship with each provider. Token rates follow the provider's published prices, while pay-as-you-go credit purchases include a 5.5% platform fee. Free models are also available with lower usage limits.

OpenRouter focuses on broad model access, provider routing, price comparison, and centralized billing. The catalog supports early model exploration and applications that route across providers. Comparing application-specific quality, defining release requirements, and monitoring production performance require a separate evaluation system. Our best unified LLM API provider comparison covers how OpenRouter compares with other LLM providers.

How Braintrust connects model access to evaluation and release control

The Braintrust Gateway provides a single API to call models using credentials managed by teams at the organization or project level. Braintrust supports major model providers, cloud platforms, and custom endpoints. Model access stays standardized without handing provider billing to an intermediary. The hosted Gateway is currently free during beta, with global and regional endpoints available for production workloads.

Gateway requests can be captured as traces and evaluated with the same datasets, scorers, and experiments used during development. Compare models against application-specific requirements, use evaluation results to control releases, and monitor production quality after deployment. Because routing and evaluation share the same system, models can be changed without rebuilding the quality process for each one.

---

Model and provider coverage

OpenRouter offers a broader ready-to-use catalog, with more than 500 models available on its website and via its Models API. Catalog filters cover provider, price, modality, and supported parameters, and an RSS feed announces newly added models. Model availability changes as providers add or remove options from the marketplace.

Braintrust Gateway covers major model providers and cloud platforms via supported APIs, including OpenAI-compatible, Anthropic, Google, and AWS Bedrock. Custom providers include self-hosted, fine-tuned, and proprietary models, and they use the same Gateway interface as supported models. Custom cost metadata keeps usage estimates accurate when Braintrust does not already have pricing information for the endpoint.

OpenRouter reaches more marketplace models faster. Braintrust brings provider-hosted and custom endpoints into a single path for routing, evaluation, and observability.

OpenAI SDK compatibility

OpenRouter and Braintrust can reuse an existing OpenAI client, but each requires different configuration changes. OpenRouter requires updating the base URL, authenticating with an OpenRouter API key, and using the model's marketplace slug. Braintrust requires a Braintrust API key and Gateway base URL, with provider credentials stored at the organization or project level.

Braintrust also translates requests and responses across supported SDK and provider combinations, so an application using the OpenAI SDK can call Claude or Gemini without separate provider libraries or additional logic for response parsing, streaming, and error handling. Check Gateway compatibility when selecting the SDK and provider combination for each model.

Upstream provider selection

OpenRouter can route a model request across multiple providers that offer the same model. Its default strategy deprioritizes providers experiencing recent outages, distributes requests among the lowest-cost eligible options using inverse-square price weighting, and uses the remaining providers as fallbacks. Request-level controls override the default selection strategy.

• sort prioritizes price, throughput, or latency.
• order, only, and ignore control provider sequence and eligibility.
• max_price excludes endpoints above a specified price.
• quantizations limits requests to selected quantization levels.
• data_collection and zdr restrict requests according to data-handling requirements.

The Activity page and generation API identify the provider that served each request and report the associated cost.

Braintrust sends requests through provider credentials configured at the organization or project level, with project credentials taking precedence. Teams can select a specific endpoint with x-bt-endpoint-name or define an ordered list of fallback providers for outages, rate limits, and server errors. The x-bt-used-endpoint response header identifies the credential that completed the request, and when failover occurs, x-bt-failover-from and x-bt-failover-to record the original and final model-provider pairs. The header trail leaves a predictable primary provider, a controlled failover order, and per-request evidence of how each response was served.

---

OpenRouter pricing vs Braintrust Gateway pricing

OpenRouter separates inference costs from its platform fee. Provider list rates pass through without a model markup, and each successful request is deducted from a prepaid credit balance. Pay-as-you-go credit purchases carry a 5.5% fee with a minimum charge of $0.80. Failed requests are not billed, so provider fallback incurs only the cost of the completed request.

Braintrust Gateway routes requests through provider credentials owned by the team, so model charges remain on existing provider accounts and retain any negotiated rates. The hosted Gateway is free during beta, with pricing to be announced before general availability. Logging and evaluating Gateway traffic are billed separately under Braintrust's usage-based pricing, which is based on processed data and scoring volume, with no per-seat fees.

BYOK on OpenRouter vs Braintrust Gateway

OpenRouter: BYOK is optional. Connect provider credentials, pay model charges through provider accounts, and continue using OpenRouter's routing layer. BYOK is not free above a monthly allowance, which OpenRouter measures by list-price inference cost. Pay-as-you-go plans include $25,000 each month, and Enterprise plans include $200,000. Above the allowance, OpenRouter charges 5% of the equivalent model and provider costs. Prioritized BYOK keys are tried before OpenRouter's shared endpoints, and fallback to shared capacity can be prevented by enabling "Always use for this provider." Data policies still apply, and BYOK inference spend counts toward workspace and guardrail budgets only when the relevant setting is enabled.

Braintrust: Provider credentials are required for every Gateway request. Organization-level credentials set the defaults across projects, and project-level provider credentials can override them when a project needs separate billing, usage isolation, or a different provider account. Applications authenticate to the Gateway with a Braintrust API key or service token, so provider credentials remain centrally configured in Braintrust and do not need to be stored in local application environments.

Caching behavior and cost impact

Braintrust Gateway caches complete responses before sending requests to the provider. Its default auto mode caches supported requests when temperature=0 or a seed is set. Override the behavior with always or never through x-bt-use-cache. Cache entries expire after seven days by default, and per-request TTLs can be set anywhere from one second to that same seven-day ceiling. The x-bt-cached, Age, and Cache-Control response headers provide the cache status and expiration details. Braintrust also encrypts cached responses with AES-GCM using a key derived from the API key, keeping results isolated by user unless the organization enables shared caching.

When the Gateway returns a cached response, the request does not reach the provider, which reduces latency and provider spend. Requests that are not served from the full-response cache can still benefit from provider-specific prompt caching. Automatic caching for deterministic requests, combined with configurable TTLs, helps rerun unchanged evaluation cases without repeatedly generating the same responses.

OpenRouter supports both response caching and provider prompt caching through its routing layer. Its response cache hashes the request body, model, API key, and streaming mode into a cache key, and can be enabled per request or through a preset. The default TTL is five minutes and can be extended to 24 hours, with cache hits bypassing the provider and carrying no usage charge. On supported models, provider prompt caching reuses prompt prefixes, with sticky routing sending related requests to the same provider endpoint to improve cache reuse. Sticky sessions expire after 10 minutes of inactivity and can be pinned explicitly with a session_id.

---

Model spend attribution

OpenRouter attributes inference spend to workspaces and API keys, with the Activity page providing usage filters for models and providers. Workspace budgets and guardrails can also enforce spending limits across teams, members, and keys, which helps organizations identify and control the account boundary responsible for each charge.

Braintrust carries cost attribution into the application trace. Each LLM span records the model, token usage, and estimated cost, so spend can be traced to a single feature or a single step inside a multi-step application. Braintrust dashboards analyze cost alongside latency and quality scores, and the comparison answers whether the more expensive model actually performs better.

What each request records

OpenRouter's Activity page and exports provide request-level usage and cost data. Each response also includes token counts, cost, and available cache details. For BYOK failures, the raw metadata lists every provider attempted during routing, along with its HTTP status code, which points directly to the account that rejected the request.

Braintrust can log each Gateway request as a span within the full application trace. Setting x-bt-parent connects the Gateway call to an existing service span, preserving the request path across service boundaries. Without distributed tracing, use x-bt-project-name to log requests directly to a Braintrust project. When x-bt-parent is set, the response includes x-bt-span-id to attach scores or user feedback after the request completes. For failed requests, x-bt-error-origin distinguishes Gateway errors from upstream provider errors, so teams can investigate reliability and quality within the application context.

---

Evaluation across model selection, release control, and production

OpenRouter's Ori Eval runs an agent against project-specific prompts, checks expected and prohibited tool calls, and uses an LLM judge to grade open-ended answers. It generates *.eval.ts files to compare models using a pinned test harness, with support for local runs and CI. Its scope centers on code-based regression testing for agent behavior.

Braintrust runs a structured evaluation process that extends from development into production:

1. Define quality standards: Create datasets from representative inputs and use custom or built-in scorers to define acceptable performance.

2. Compare models and prompts: Experiments preserve the inputs, outputs, scores, latency, and cost from every run. Model and prompt decisions then rest on recorded numbers rather than recollection.

3. Enforce release requirements: Run evaluations in CI and prevent changes from progressing when results fall below approved thresholds.

4. Evaluate production behavior: Online scoring applies the established quality criteria to production traces, and failed interactions can become evaluation cases for future releases.

Ori Eval covers agent regression testing within a codebase; Braintrust applies the same quality criteria from model selection through release gating to production scoring.

How rate limits, failover, and provider variance affect reliability

Rate limits: OpenRouter's free model variants share an account-wide limit of 50 requests per day, increasing to 1,000 after the account has purchased at least $10 in credits. Paid variants carry no daily platform cap, though both free and paid requests share a 20-requests-per-minute limit and upstream provider limits still apply. Braintrust Gateway uses provider credentials owned by the team, so requests remain subject to the limits of the corresponding provider accounts.

Fallback behavior: OpenRouter automatically routes around unavailable endpoints offering the selected model. Configure model-level fallbacks via a prioritized models array, allowing another model to handle requests that are affected by downtime, rate limits, moderation refusals, or context-length validation errors. Braintrust defines an ordered list of credentials via the x-bt-fallback-providers header. The Gateway retries provider unavailability, 429 rate limits, and 5xx errors, but does not retry authentication failures, validation errors, or other provider-side 4xx errors. Gateway response caching is disabled for failover requests, so x-bt-cached returns N/A.

Provider consistency and visibility: A single OpenRouter model slug can route to provider endpoints with different latency, throughput, uptime, pricing, and quantization levels. Restricting or ordering providers, setting performance preferences, and limiting accepted quantization levels all narrow the variance. Braintrust preserves a team-defined primary provider and fallback sequence, with x-bt-used-endpoint, x-bt-failover-from, and x-bt-failover-to identifying the credential and model-provider path that completed the request.

For teams selecting an LLM router, OpenRouter emphasizes automatic resilience across marketplace capacity, while Braintrust provides operational control to define the provider order and verify the serving path for each request.

Deployment options and data residency controls

Braintrust Gateway provides global latency-based routing and dedicated endpoints in US East, US West, EU West, and Asia Pacific Southeast. Gateway routing remains separate from data storage, so logged requests are written to the organization's configured data plane. Braintrust supports three data-plane deployment options. SaaS runs in Braintrust-managed infrastructure, BYOC runs in the customer's cloud under Braintrust management, and self-hosted runs in the customer's cloud under its own management. BYOC and self-hosted deployments require Enterprise, and both keep sensitive AI data within the customer's cloud account and region.

OpenRouter operates as a hosted routing service. Restrict eligible providers through data-collection policies and zero-data-retention controls applied at the account, guardrail, or request level. Enterprise customers can also use in-region routing in the EU and the US, which decrypts and processes prompts and completions entirely within the selected region and routes only to providers operating there. SSO, managed policy enforcement, and administrative controls are also reserved for Enterprise, and OpenRouter reports SOC 2 Type 2 and GDPR compliance.

OpenRouter provides regional processing and provider-policy controls within a hosted service. Braintrust supports broader data residency and operational ownership requirements by providing options to choose where evaluation and production data is stored and who operates the data plane.

---

Is OpenRouter suitable for production LLM applications?

OpenRouter can support production applications that prioritize broad model access and flexible routing across marketplace providers. Workspaces can separate staging and production configurations, guardrails can enforce usage and data policies, and Broadcast can send request traces to external observability systems. Ori Eval also provides code-based model comparisons and CI checks.

Production suitability depends on the consistency and quality governance the application requires. A model slug can reach different upstream endpoints unless providers are restricted or ordered, which means reproducibility depends on explicit routing policies. OpenRouter exposes the serving path and can export traces, but teams must connect those records and Ori results to the release and monitoring processes that determine whether application quality remains acceptable.

Braintrust provides stronger release control when Gateway traces, evaluation results, and production-quality monitoring are required in a single workflow.

Running OpenRouter and Braintrust together

Use OpenRouter to explore models, then route the selected model through the Braintrust Gateway when it becomes part of a production feature. OpenRouter provides a single API for testing models across its marketplace, and Braintrust provides a controlled production call path with provider routing connected to the quality requirements used for release decisions.

OpenRouter can also continue to handle production calls while Braintrust traces requests through its OpenRouter integration. Logging OpenRouter and Gateway requests to the same Braintrust project allows applying shared scorers and dashboards across both, preserving marketplace access within a single evaluation and production monitoring process.

---

Which teams each product suits

OpenRouter suits teams that prioritize broad model access and consolidated billing. Common use cases include early-stage products still selecting models, applications that switch among providers for cost or availability, and agencies supporting clients without direct provider accounts.

Braintrust serves teams that make AI quality a release requirement. Evaluating models, prompts, and application changes against defined quality standards provides engineering, product, and subject-matter experts with clear evidence to decide what is ready to ship. Braintrust also connects application-level cost with quality results to account for performance and spend in each release decision.

Choosing between OpenRouter and Braintrust

Four criteria decide which product carries production traffic.

Model breadth: A broad, frequently changing model catalog points toward OpenRouter's marketplace. Once production uses a known set of providers, catalog breadth carries less weight, and Braintrust's evaluation and release controls become more relevant.

Provider accounts: OpenRouter credits allow teams to use supported models without opening an account with every provider, and BYOK remains optional. Braintrust Gateway requires provider credentials owned by the team, so inference spend, rate limits, and provider terms remain attached to those accounts.

Release requirements: OpenRouter's Ori Eval supports code-based model comparisons and agent tests. When evaluation results must prevent a regression from reaching production, Braintrust connects development evaluations with release requirements and production scoring.

Data residency: OpenRouter provides hosted routing with provider data policies and in-region processing in the EU and the US. Braintrust supports SaaS, BYOC, and self-hosted data planes, although BYOC and self-hosting require Enterprise.

OpenRouter centers broad model access and shared credits, whereas Braintrust connects provider routing to evaluation and release control. Teams weighing other options can compare the field in our OpenRouter alternatives guide.

Start free with Braintrust to evaluate application changes before they reach production.

---

Braintrust vs OpenRouter FAQs (2026)

What is the main difference between OpenRouter and Braintrust?

OpenRouter combines model access and billing across a large provider marketplace. Braintrust Gateway uses provider accounts controlled by the team and connects routed requests with traces, evaluation results, and production-quality data.

Is Braintrust an alternative to OpenRouter?

Braintrust can replace OpenRouter on the production request path when a team already manages its provider accounts. Braintrust Gateway standardizes access to supported providers and keeps routing connected to evaluation and observability. It does not replicate OpenRouter's shared-credit marketplace, so teams that need both capabilities can continue routing through OpenRouter and send the resulting traces to Braintrust.

Does OpenRouter have built-in evals?

OpenRouter now includes Ori Eval, which creates code-based agent tests for comparing models and can run them locally or in CI. Ori Eval stays inside the codebase, comparing models and checking agent behavior against a pinned harness. Braintrust extends evaluation into release enforcement and production scoring, allowing defined quality standards to determine what ships and assess live traffic.

Does the Braintrust Gateway add a markup on tokens?

Braintrust Gateway adds no markup to provider token costs because each request uses credentials configured by the team, and the provider bills inference at the corresponding account rate. The hosted Gateway remains free during beta, with logged data and scores covered separately.

Should a production LLM application use OpenRouter or Braintrust?

OpenRouter works well when broad model coverage and consolidated inference billing are the priorities. Braintrust is better suited to teams that manage their own provider credentials and need evaluation results to govern releases and monitor production quality. Applications that require both can use OpenRouter for model access and Braintrust for evaluation and observability.