Technology Radar
agentgateway is a high-performance gateway for LLM-related traffic, hosted by the Linux Foundation. It supports LLM inference, MCP and A2A communication through a common protocol in both standalone and Kubernetes versions. Core features include cost- and latency-based routing, fallbacks, spending limits, guardrails, MCP tool control and OpenTelemetry observability.
We’re particularly interested in the Kubernetes version because of its deep integration with self-hosted LLM infrastructure. It can use Kubernetes metrics to route requests to a GPU-backed replica that already has the relevant context cached in memory or to a less utilized server based on actual GPU usage. These capabilities make agentgateway worth evaluating for organizations running self-hosted inference stacks.
