LiteLLM

Proxy Server to call 100+ LLMs in the OpenAI format

An open alternative to ChatGPT

Author: Berrie AI, Inc · License: MIT

Version: 1.53.2 · Updated 2 weeks ago

About

Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.]

preview

Translate inputs to provider's completion, embedding, and image_generation endpoints
Consistent output, text responses will always be available at ['choices'][0]['message']['content']
Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - Router
Set Budgets & Rate limits per project, api key, model LiteLLM Proxy Server (LLM Gateway)

View environment variables

PROJECT: litellm
DOMAIN: localhost
LITELLM_MASTER_KEY: "sk-1234" # Your master key for the proxy server. Can use this to send /chat/completion requests etc
LITELLM_SALT_KEY: "sk-XXXXXXXX" # Can NOT CHANGE THIS ONCE SET - It is used to encrypt/decrypt credentials stored in DB. If value of 'LITELLM_SALT_KEY' changes your models cannot be retrieved from DB
OPENAI_API_KEY: ""
OPENAI_API_BASE: ""
COHERE_API_KEY: ""
OR_SITE_URL: ""
OR_APP_NAME: "LiteLLM Example app"
OR_API_KEY: ""
AZURE_API_BASE: ""
AZURE_API_VERSION: ""
AZURE_API_KEY: ""
REPLICATE_API_KEY: ""
REPLICATE_API_TOKEN: ""
ANTHROPIC_API_KEY: ""
INFISICAL_TOKEN: ""

Open WebUI

User-friendly AI Interface
★ 51.7k LLM AI RAG Chatbot
AnythingLLM

The all-in-one AI app you were looking for
★ 27.8k LLM AI RAG Chatbot