August

August brings a native OCR endpoint, a Headroom plugin for reducing input token costs, and OAuth Client Credentials for keyless upstream authentication. We’ve also added Deepgram for speech workloads, new guardrail partners, and shipped improvements across Anthropic, Bedrock, and Vertex AI. See what’s new:

​

Summary

​

Gateway

​

Native OCR Endpoint

OCR requests now get the full gateway treatment — retries, fallbacks, load balancing, caching, logging, and pricing — through a unified POST /v1/ocr endpoint. Supports Mistral AI, Azure AI Foundry, and Vertex AI, with cost tracked per page processed. Learn more: OCR API

​

OAuth Client Credentials for Upstream Auth

Upstream providers now authenticate without a static API key. The gateway acquires and caches a bearer token from your configured token endpoint using the OAuth Client Credentials grant and forwards it upstream automatically. Learn more: OpenAI integration

​

MCP Tool Call Rate Limits

Tool invocation volume is now governable—rate limits apply to MCP tool calls just like they do to LLM requests. Set limits per config to prevent runaway agent loops or unexpected tool usage spikes. Learn more: MCP rate limits

​

Guardrails

​

Headroom Plugin

Input token costs go down without changing your prompts. The Headroom plugin compresses request context before it reaches the model, configured through the standard guardrails workflow. Learn more: Headroom plugin ** Other updates:**

• Singulr — now available as a guardrail partner for request and response scanning
• Header forwarding — specific client headers (e.g. x-session-id, traceparent) now flow through to guardrail upstream providers. Credential headers are never forwarded.

​

Models and providers

​

New providers

• Deepgram — speech-to-text and text-to-speech, with full observability and reliability features

​

Qwen

• Responses API and Messages API now supported, along with additional non-OpenAI parameters

​

Databricks

• Responses API now supported for Databricks-served models

​

Anthropic

• New beta parameters now passed through: inference_geo, diagnostics, fallbacks, and context_management
• thinking_tokens now correctly mapped to OpenAI’s reasoning_tokens for unified token tracking

​

Bug fixes and improvements

• MCP Registry requests now proxied through the gateway
• Meshy and Tripo3D: credit-based cost tracking added

​

Support