Skip to content

feat(models): add [LiteLLM AI Gateway]#1850

Open
RheagalFire wants to merge 2 commits intodottxt-ai:mainfrom
RheagalFire:feat/add-litellm-provider
Open

feat(models): add [LiteLLM AI Gateway]#1850
RheagalFire wants to merge 2 commits intodottxt-ai:mainfrom
RheagalFire:feat/add-litellm-provider

Conversation

@RheagalFire
Copy link
Copy Markdown

@RheagalFire RheagalFire commented Apr 20, 2026

Description

  • Add LiteLLM AI Gateway as a native model backend, giving outlines users structured generation across 100+ LLM providers (OpenAI, Anthropic, Google, Cohere, Mistral, Bedrock, Azure, Ollama, and more) through a single from_litellm() factory function.
  • Adds LiteLLM (sync) and AsyncLiteLLM (async) model classes with streaming support.
  • LiteLLMTypeAdapter inherits from OpenAITypeAdapter since LiteLLM uses the same response_format structure.
  • Added litellm as optional dependency in pyproject.toml.

Type of change

  • New features (non-breaking change).
  • Bug fix (non-breaking change).
  • Breaking change (fix or feature that would cause existing functionality not to work as expected).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant