Unify 100+ LLM APIs with litellm, a Python SDK and proxy server. Ideal for founders needing streamlined AI gateway solutions.
intermediateโฑ 1-2 hours๐ต Free (self-hosted)
51,973 stars9,287 forksPythonQuality 9/10Updated 6/29/2026100% free ยท open source
What it is
Makes it easy to call 100+ LLM APIs in OpenAI format from Python, with cost tracking, and more
What you can make with it
Automations like: when a new customer signs up, use an LLM to generate a personalized onboarding email in Notion.
How it helps
Saves development time when using multiple LLM APIs, and helps track costs associated with using these APIs.
Real use case example
"A solo developer building a customer support chatbot can use litellm to easily integrate with multiple LLM APIs, like OpenAI and Anthropic, to create more accurate and personalized responses. To do this, they simply install the Python SDK, set up their APIs, and write a simple script to call the LLMs for every new chat."
If you're new
Pick this up once you're familiar with Python and basic AI concepts.
If you're senior
Senior engineers use litellm when building large-scale chatbots, or other applications that require integrating multiple LLM APIs.
Common confusion cleared up
Don't assume litellm is just another OpenAI wrapper - it supports many other LLM APIs and has features like cost tracking.
Best inside these AI tools
Claude DesktopClaude Code
Pairs with
Claude DesktopClaude CodeCursor
Why we list it on WorkflowStacks: Free and open-source, allowing users to save cost and easily integrate with their own apps.
What it does
Litellm is a Python SDK and proxy server that simplifies calling over 100 LLM APIs from various providers, including OpenAI, Azure, and Google Vertex AI, with features like cost tracking, guardrails, load balancing, and logging.
Install / run
pip install litellm
When to use it
โขWhen you need to integrate multiple LLM APIs into your application and want to manage them through a single interface
โขWhen you want to track and control costs associated with LLM API calls across different providers
โขWhen you need to implement guardrails and load balancing to ensure reliable and efficient LLM API usage
Quick start
1Clone the Litellm GitHub repository using `git clone https://github.com/BerriAI/litellm.git`
2Create a configuration file `config.yaml` to specify your LLM API credentials and settings, as shown in the example `config.example.yaml` file
3Run the Litellm proxy server using `litellm serve --config config.yaml`
4Use the Litellm Python SDK to call LLM APIs, such as `from litellm import Client; client = Client('openai'); response = client.complete(prompt='Hello, world!')`
5Verify the cost tracking and logging features by checking the `logs` directory and the `costs` table in the database
Ready-to-paste prompt
from litellm import Client; client = Client('openai'); response = client.complete(prompt='Write a short story about a character who discovers a hidden world.', max_tokens=512)
Heads up: You need to replace the placeholder API keys and credentials in the `config.example.yaml` file with your actual credentials from the LLM providers, such as OpenAI, Azure, or Google Vertex AI, to use Litellm successfully
Saves to your device
Topics
ai-gateway
anthropic
azure-openai
bedrock
gateway
langchain
litellm
llm
llm-gateway
llmops
mcp-gateway
openai
openai-proxy
vertex-ai
What's inside โ free to inspect
No purchase needed
Read the entire source before you build โ unlike paid marketplaces that hide it behind a buy button.