llm-gateway

gitblame-hemanth/llm-gateway

★ 0 stars Python AI/LLM Updated today

Unified LLM gateway with multi-provider routing, caching, MCP server, and observability

View on GitHub → 🔍 Audit Wallet Slippage →

Quick Install

Copy the config for your editor. Some servers may need additional setup — check the README.

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "llm-gateway": {
      "command": "uvx",
      "args": [
        "llm-gateway"
      ]
    }
  }
}

Or install with pip: pip install llm-gateway

README Excerpt

A unified, OpenAI-compatible API gateway that routes LLM requests across multiple providers with automatic fallback, cost tracking, caching, and rate limiting. Point any OpenAI SDK client at the gateway and transparently access OpenAI, Anthropic, Google, AWS Bedrock, Azure OpenAI, and local models through a single endpoint.

Tools (11)

ANTHROPIC_API_KEYAWS_REGIONAZURE_OPENAI_API_KEYAZURE_OPENAI_DEPLOYMENTAZURE_OPENAI_ENDPOINTGATEWAY_API_KEYSGATEWAY_ENVGCP_PROJECT_IDGCP_REGIONOPENAI_API_KEYREDIS_URL

llm-gateway

Quick Install

README Excerpt

Tools (11)

Related AI/LLM Servers