关于 LiteLLM
使用OpenAI格式调用所有LLM API [Bedrock、Huggingface、VertexAI、TogetherAI、Azure、OpenAI、Groq等]
功能特点
- 将输入转换到提供商的
completion、embedding和image_generation端点 - 一致的输出，文本响应将始终位于
['choices'][0]['message']['content'] - 跨多个部署（如Azure/OpenAI）的重试/故障转移逻辑 - 路由器
- 为每个项目、API密钥、模型设置预算和速率限制 LiteLLM代理服务器（LLM网关）
环境变量
查看环境变量
- PROJECT
- litellm
- DOMAIN
- stack.localhost
- LITELLM_MASTER_KEY
- "sk-1234" # Your master key for the proxy server. Can use this to send /chat/completion requests etc
- LITELLM_SALT_KEY
- "sk-XXXXXXXX" # Can NOT CHANGE THIS ONCE SET - It is used to encrypt/decrypt credentials stored in DB. If value of 'LITELLM_SALT_KEY' changes your models cannot be retrieved from DB
- OPENAI_API_KEY
- ""
- OPENAI_API_BASE
- ""
- COHERE_API_KEY
- ""
- OR_SITE_URL
- ""
- OR_APP_NAME
- "LiteLLM Example app"
- OR_API_KEY
- ""
- AZURE_API_BASE
- ""
- AZURE_API_VERSION
- ""
- AZURE_API_KEY
- ""
- REPLICATE_API_KEY
- ""
- REPLICATE_API_TOKEN
- ""
- ANTHROPIC_API_KEY
- ""
- INFISICAL_TOKEN
- ""