The A3M Router team has officially shipped v2.0.0, the most significant update since the project's inception. What began as a lightweight routing library has been transformed into a comprehensive AI Gateway.
A standout feature is the new OpenAI-Compatible Proxy Server. By running npx a3m-router serve, developers can spin up a proxy on localhost:8787. This serves as a drop-in replacement for any OpenAI SDK. By simply updating the base_url and setting the model to auto, the gateway handles provider keys and intelligent routing across 39 different AI vendors automatically.
The update also introduces a Real-Time Dashboard accessible via the local server. It provides live insights into request volumes, provider status (online/offline), detailed request logs with routing decisions, and a granular cost breakdown by provider.
For the LangChain ecosystem, the new A3MChatModel adapter offers a seamless replacement for ChatOpenAI, supporting streaming, tool calling, and structured outputs. Security is bolstered by a built-in Guardrails Engine that detects prompt injections, PII (emails, API keys, etc.), harmful content, and performs language detection for smarter routing.
To optimize performance and cost, A3M v2.0 features a Semantic Cache. Using n-gram similarity thresholds rather than external embedding APIs, it can identify and serve semantically similar queries from the cache, significantly reducing API latency and costs. Finally, the new Cost Analytics module allows developers to track and analyze usage metrics in real-time, ensuring full transparency over AI spend.
