Use Any LLM Provider with LiteLLM | Use ChatGPT, Claude, Gemini, Ollama with One API

Switching between large language model (LLM) providers can break production systems when code depends on a single vendor’s SDK. LiteLLM is presented as a practical fix: it provides one unified interface that can route requests across multiple providers—so teams can swap models like OpenAI, Google Gemini, Anthropic, or local Ollama without rewriting their application logic.

The walkthrough starts by installing and wiring LiteLLM alongside supporting libraries such as Pydantic (for schema validation) and NumPy (used for parsing tool docstrings). The core pattern is consistent with familiar LLM SDK usage: define an API key per provider, craft a messages list, and call a single completion function while specifying the target model name. A test prompt (“what is tarning in one sentence”) is sent to GPT 4.1 mini through LiteLLM, and the response returns both the generated text and detailed usage metrics like prompt tokens, completion tokens, and total tokens.

The same call pattern then targets Google’s Gemini 2.5 models (example: Gemini 2.5 Flash). The interface remains effectively identical, including the ability to pass provider-specific parameters—such as disabling “thinking” for a reasoning model—while still receiving a similarly structured response object. The practical takeaway is that model switching becomes mostly a matter of changing the model identifier and a few parameters, not the surrounding application code.

Beyond basic text generation, the transcript emphasizes two workflow features that matter for building reliable AI systems: structured outputs and tool calling. For structured outputs, LiteLLM is paired with Pydantic to force the model to return JSON that matches a defined schema. A sentiment classification example defines a Pydantic model with fields for sentiment (negative/neutral/positive) and reasoning. The model output is validated as JSON and converted into a typed object, making downstream logic safer than parsing free-form text.

For tool calling, the transcript demonstrates an agent-style flow. A function named “estimate_house_price” is defined with typed parameters (square meters, number of bedrooms, and a boolean for whether the location is inexpensive). LiteLLM converts the function into a tool definition, sends it to the model with tool_choice set to auto, and receives one or more tool calls. The code then iterates through returned tool calls, matches the function name to the available implementation, parses the JSON arguments, executes the Python function, and appends the tool result back into the message history. A final model call produces a user-facing answer—estimated at about $650,000 for a 3-bedroom, 250-square-meter house in San Jose.

The transcript also frames LiteLLM as an operational safeguard: it can act as a fallback during outages or latency spikes. If OpenAI is slow or unavailable, the same application can route failed requests to Gemini or Anthropic until the primary provider recovers. The result is a more resilient architecture for evaluations, agent workflows, and production deployments that depend on multiple LLM vendors.