Braintrust vs Portkey: gateway with evals vs gateway with guardrails

---

Quick comparison: Braintrust vs Portkey

Compare model and application changes before release. Start free with Braintrust.

---

What Braintrust is and where the Gateway fits

Braintrust connects evaluation and production observability, allowing application changes to be assessed against consistent quality standards before and after release. Development results and production traces remain part of the same evaluation process, providing product and engineering teams with clear evidence to improve application quality.

The Braintrust Gateway adds model routing to Braintrust's evaluation and observability platform, giving applications a single endpoint for supported providers and connecting routed requests with the traces and evaluation results used to assess application quality. The hosted Gateway is currently in beta and free during the beta period.

What Portkey is and what the Palo Alto acquisition changed

Portkey began as an AI gateway for routing model requests through an OpenAI-compatible endpoint and applying runtime controls through configuration. Palo Alto Networks completed its acquisition of Portkey in May 2026 and used its technology as the foundation for Prisma AIRS AI Gateway, which reached general availability in July 2026 as part of the Palo Alto Networks enterprise AI security platform.

Prisma AIRS AI Gateway emphasizes centralized governance and runtime security across models, agents, and MCP servers. Portkey's open-source gateway remains available for self-hosting, but organizations seeking the managed enterprise offering now purchase it through Palo Alto Networks.

---

Model coverage and SDK compatibility

The Braintrust Gateway supports models across OpenAI-compatible, Anthropic, Google, and AWS Bedrock APIs. Supported providers include OpenAI, Anthropic, Google, AWS, Azure, Mistral, and Vertex AI, and custom providers extend access to self-hosted, fine-tuned, and proprietary endpoints.

Cross-SDK compatibility allows an application to keep its existing client when changing providers. An OpenAI client can call Claude, an Anthropic client can call Gemini, and a Google client can call GPT by updating the base URL and model name. The Gateway also supports embeddings through the OpenAI-compatible /embeddings endpoint and Gemini's native embedContent endpoints.

Portkey reaches a broader catalog through an OpenAI-compatible endpoint, citing access to more than 1,600 models. Palo Alto Networks also describes access to more than 3,000 LLMs, MCP servers, and tools, but that total combines several asset types and should not be read as a model-only count. The larger catalog is most relevant to organizations evaluating many open-weight models or less common providers.

Portkey leads on catalog breadth. Braintrust covers major providers and custom endpoints, with cross-SDK compatibility that reduces the code changes required to switch among them.

Routing and failover

The Braintrust Gateway supports ordered provider failover for compatible models. Set the x-bt-fallback-providers header to an ordered list of provider credential names. If the primary provider is unavailable, returns a rate-limit error (429), or fails with a 5xx, the Gateway retries with each configured fallback in order. Authentication, validation, and other client errors do not trigger failover, and response caching is disabled for failover requests. When a fallback succeeds, the x-bt-failover-from and x-bt-failover-to response headers identify the original and final model-provider pairs.

Portkey provides a broader routing policy layer. Its configurations can combine fallback chains, weighted load balancing, retries, timeouts, canary testing, circuit breakers, and conditional rules based on request parameters or metadata. These controls support applications that route traffic differently across models, providers, regions, or request types.

Response and semantic caching

Braintrust Gateway response caching uses the x-bt-use-cache header with three modes:

• auto caches requests on supported paths when temperature=0 or a seed value is present.
• always caches every request on a supported path.
• never prevents the cache from being read or written.

Cached responses expire after one week by default. The x-bt-cache-ttl header sets a per-request TTL between one second and seven days, and supported Cache-Control directives take precedence. Results are encrypted with AES-GCM using a key derived from the caller's API key, which scopes cached data to an individual user unless the organization enables shared caching.

Portkey supports simple caching for exact-match requests across all plans. Its semantic cache, available on select Enterprise plans, checks for an exact match first, then applies cosine similarity on a miss. Semantic matching expands cache reuse when users phrase the same request differently.

Guardrails and request-time enforcement

Portkey runs guardrails as before-request and after-request hooks, allowing configured checks to inspect prompts before model calls and responses before they return to the application. Coverage includes more than 20 deterministic checks, LLM-based detection for conditions such as gibberish and prompt injection, partner integrations, and custom webhooks.

For streaming responses, input guardrails run before the first token and can block the request. Output guardrails evaluate the complete response after streaming finishes. Since the tokens have already been delivered, output results are informational and do not automatically trigger denial, fallback, or retry.

Braintrust Gateway does not block prompts or responses in the request path. Online scoring evaluates logged production traces asynchronously without adding latency, and guardrail evaluation measures whether runtime controls catch prohibited behavior without rejecting valid interactions.

Governance and access controls

Portkey has moved provider governance from Virtual Keys to its Model Catalog, where existing virtual keys now appear as AI Providers and continue to work with existing configs. A single Portkey API key can access approved providers without exposing provider credentials in application code. Administrators set model access, budgets, rate limits, and workspace permissions through policy, and review administrative activity through audit logs.

Braintrust governs access through permission groups that operate at the organization, project, and individual object levels. The Pro plan includes the Owner, Engineer, and Viewer groups. Enterprise adds custom permission groups, audit logs, and SAML or OIDC SSO, as detailed in Braintrust plans and limits. Service tokens inherit assigned permissions, allowing automated systems to receive narrowly scoped access.

Where Portkey enforces limits at request time through model restrictions, spending caps, and quotas, Braintrust governs who can reach evaluation resources, production data, and the platform itself. Cost alerts can flag expensive requests or spending increases, but the Gateway does not enforce per-key budgets or request quotas.

Prompt management and versioning

Portkey's Prompt Engineering Studio organizes prompts in a shared library with folders and access controls. Version history supports rollback, partials reuse common instructions, and the playground compares prompt or model variants side by side. Saved prompts are available to applications through Portkey's API.

Braintrust supports creating prompts in the UI or code and records every saved change as a new version. Applications may pin an exact version or load the version assigned to a development, staging, or production environment. Playgrounds run prompt versions against datasets and scorers, providing direct comparisons of output quality, latency, and cost. Those scores are what a team points to when deciding which version ships.

Observability and trace analysis

Portkey records more than 40 request-level data points, including cost, latency, token use, cache status, errors, and guardrail results. Its observability interface combines logs and traces with feedback, custom metadata, filters, and alerts. Enterprise deployments can export analytics and experimental GenAI traces to OpenTelemetry-compatible collectors, placing gateway activity alongside application telemetry.

Braintrust keeps every logged Gateway call as an addressable span. Setting x-bt-parent nests the call within an existing service trace, and the returned x-bt-span-id identifies the span, allowing updateSpan to attach scores, metadata, or feedback after the request completes. Setting x-bt-project-name by itself logs and traces requests to the selected project.

Topics classify logs by Task, Sentiment, Issues, and custom facets. A filtered set of traces can become a targeted evaluation dataset, which turns a recurring production failure into a repeatable test case for any proposed fix.

Evaluation and release control

Braintrust connects evaluation across development and production. Experiments run application versions against datasets and use code-based or LLM-as-a-judge scorers, preserving each run as an immutable record to measure improvements and regressions. Online scoring applies the same scorers to production traces after the fact, so the standards a team sets before release continue to run against live traffic.

CI/CD evaluation runs experiments on pull requests, reports results, and can gate merges based on custom pass/fail criteria. Production failures can become permanent dataset cases, and Loop lets contributors describe issues in natural language and generate datasets or scorers without writing evaluation code.

Portkey supports prompt and model comparisons, LLM-as-a-judge checks, and batch evaluations built from its prompt, guardrail, and batch-processing features. These capabilities help validate prompt outputs and runtime controls. Braintrust extends evaluation into release decisions by connecting production evidence, reproducible experiments, and CI requirements on a single platform.

Deployment options, self-hosting, and data residency

Braintrust supports SaaS, BYOC, and self-hosted deployments. SaaS is available on every plan, with Braintrust operating both the control and data planes. The other two require Enterprise. Under BYOC, the data plane sits in the customer's cloud account while Braintrust handles upgrades, scaling, monitoring, and incident response; under self-hosting, the customer runs the data plane itself using published Terraform modules for AWS, Google Cloud, or Azure.

Both customer-cloud options keep sensitive AI data, including traces, datasets, prompts, and experiment logs, within the customer's cloud account and selected region. Braintrust continues to operate the control plane for the UI, authentication, and platform management. Gateway routing remains independent of the selected data plane region, and logged Gateway requests are written to the organization's configured data plane.

Portkey's LLM gateway can be self-hosted without request limits. Its Enterprise offering also supports managed SaaS, hybrid deployments in which the data plane remains within the customer's VPC while Portkey hosts the control plane, and fully air-gapped deployments that keep every component within the customer's network.

For regulated workloads, Braintrust is SOC 2 Type II compliant and supports DPAs, with BAAs available on Enterprise. Portkey lists SOC 2 Type 2, ISO 27001, GDPR, HIPAA, and BAA support for its Enterprise offering.

Pricing and usage costs

Braintrust offers a free Starter plan and a Pro plan starting at $249 per month, with custom pricing available for Enterprise. Every plan includes unlimited users and core evaluation resources, so costs do not increase when more people participate in evaluation and review. SaaS is available across all three plans, whereas BYOC and self-hosted deployments require the Enterprise plan.

Portkey's public self-service pricing is based on recorded gateway activity, with usage allowances and retention periods varying by plan. Managed Prisma AIRS AI Gateway pricing requires a custom quote from Palo Alto Networks.

Braintrust generally offers better value when gateway traffic is part of an ongoing evaluation program across development and production. Portkey's request-based self-service plans may be more economical for gateway-only workloads, although they cannot be used to estimate the managed Prisma AIRS product, which requires a custom quote. Final costs depend on usage, but Braintrust becomes more cost-effective as evaluation expands across teams because it does not charge separately for each user or evaluation resource.

Product status and ownership considerations

The hosted Braintrust Gateway is in beta and already carries production traffic, with uptime tracked on the Braintrust status page. Its final pricing and general-availability date have not been announced. Braintrust's established evaluation and observability capabilities do not depend on Gateway adoption, allowing applications to use them with direct provider connections or another gateway.

Portkey is now owned by Palo Alto Networks, and its technology underpins the generally available Prisma AIRS AI Gateway. The managed enterprise product is purchased through Palo Alto Networks, and the open-source Portkey gateway remains available under an MIT license. Organizations should confirm whether a planned deployment uses Portkey's self-service product, the open-source gateway, or the Prisma AIRS AI Gateway, and the associated support and commercial terms.

---

Choosing between Braintrust and Portkey

Gateway control: Portkey fits applications that need conditional routing, enforced budgets and rate limits, or synchronous input guardrails. Braintrust handles provider failover and caching but stops short of blocking prompts or responses in the request path.

Quality control: When evaluation results must determine whether a model, prompt, or application change is ready for production, Braintrust is the stronger fit. Release requirements carry the agreed quality bar into the merge process, and production evidence feeds back into the next round of evaluations.

Model coverage: Portkey's catalog runs wider, reaching more open-weight models and less-common providers. The Braintrust Gateway covers major providers and custom endpoints, and its cross-SDK compatibility means a team can switch models without replacing its existing client.

Deployment: Every Braintrust plan includes SaaS; BYOC and self-hosted data planes require the Enterprise plan. Portkey's open-source gateway is free to self-host, and the managed enterprise product comes through Palo Alto Networks.

Cost structure: Braintrust generally offers better value for teams running evaluation across development and production because its plans include unlimited users and core evaluation resources. Portkey's request-based self-service plans may cost less for gateway-only workloads, while managed Prisma AIRS AI Gateway requires a custom quote.

Start free with Braintrust and use evaluation results to make production release decisions.

---

Using Braintrust and Portkey together

A combined setup is useful when Portkey already manages request-time controls, and the team wants Braintrust to govern application quality. Portkey can remain in the request path while the application sends complete traces to Braintrust through its SDK or OpenTelemetry integration. Each trace can include the Portkey-routed model call alongside retrieval, tool use, and application logic, so adopting Braintrust does not require replacing the existing gateway.

Application tracing should be the primary source because it provides the context required for evaluation. Exporting the same model call from Portkey can create duplicate traces and usage records, so gateway log export is better reserved for cases where application-level tracing is unavailable.

A separate Portkey deployment is unnecessary when the Braintrust Gateway already meets the application's routing, failover, and caching requirements. Teams still selecting their gateway can review the best LLM gateways for developers and the broader AI gateway comparison, or see the field of Portkey alternatives.

Why Braintrust leads for quality-critical AI products

Braintrust makes model quality an enforceable part of the release process. Engineering, product, and subject-matter reviewers can define expected behavior, assess proposed changes against shared criteria, and use evaluation results to decide what reaches users. Production evidence strengthens later evaluations, keeping quality standards consistent from development through release.

Teams at Notion, Stripe, Vercel, Instacart, Zapier, and Ramp use Braintrust for production AI evaluation. After introducing systematic evaluation with Braintrust, Notion increased its issue-resolution rate from 3 to 30 fixes per day.

Start free with Braintrust or get a personalized demo.

---

FAQs about Braintrust vs Portkey (2026)

Is Braintrust Gateway production-ready while it is in beta?

Although the hosted Braintrust Gateway remains in beta, it is ready to route live application traffic in production. Teams can adopt the Gateway now, while feature refinement continues before general availability and pricing are announced.

Does Braintrust offer runtime guardrails that block unsafe responses?

Braintrust Gateway does not block prompts or responses during a request. Teams that need synchronous blocking usually pair a request-time layer with Braintrust and then use online scoring to verify that the layer is catching what it should.

Is Portkey still available after the Palo Alto Networks acquisition?

Portkey remains available, and its open-source gateway continues under the MIT license. Palo Alto Networks also uses Portkey technology in Prisma AIRS AI Gateway, its managed enterprise offering. Buyers should confirm whether they are adopting the open-source gateway, Portkey's self-service product, or Prisma AIRS, because deployment, support, and commercial terms differ.

Can you self-host either Braintrust or Portkey?

Both Braintrust and Portkey support deployment in customer-controlled infrastructure under different conditions. Braintrust offers BYOC and self-hosted data planes on Enterprise, with deployment options for AWS, Google Cloud, and Azure. Portkey's MIT-licensed gateway can be self-hosted independently, although managed control-plane capabilities and vendor support depend on the selected commercial plan.

Do I need an AI gateway with built-in evaluation or one with guardrails?

Use a gateway with inline guardrails when the application must block unsafe, sensitive, or policy-violating prompts and responses before they reach users or trigger actions. Choose built-in evaluation when model, prompt, and application changes must meet defined quality standards before release and must continue to perform reliably in production. Many production applications need both because guardrails enforce rules on individual requests, and Braintrust evaluates the complete application, including whether the guardrails themselves are tuned correctly.

What is a good Portkey alternative with evals?

Braintrust is the strongest Portkey alternative for teams that need gateway traffic connected to structured evaluation and release control. OpenAI-compatible applications can route model calls through the Braintrust Gateway by changing the base URL to https://gateway.braintrust.dev and using a Braintrust API key. Braintrust then adds production traces to reproducible evaluations and release requirements. Teams that rely on Portkey's inline guardrails, budget enforcement, or rate limits will still need a request-time layer alongside Braintrust.