LiteLLM

Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.]

Features

• Translate inputs to provider's completion, embedding, and image_generation endpoints
• Consistent output, text responses will always be available at ['choices'][0]['message']['content']
• Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - Router
• Set Budgets & Rate limits per project, api key, model LiteLLM Proxy Server (LLM Gateway)