feat(llm): add openai_compatible provider#165
Merged
Conversation
added 2 commits
July 6, 2026 20:02
This commit adds type hints and uses specific OpenAI message parameters for chat completions, improving code clarity and ensuring proper usage of the OpenAI API.
This commit introduces support for OpenAI-compatible endpoints, including new tests, configuration options, and documentation updates. It also ensures that the `openai_compatible` provider correctly handles `base_url` and `api_key` with credential references.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#167
Summary
New
openai_compatibleLLM provider so[summary]/[description]can target any OpenAI Chat Completions-compatible endpoint (DeepSeek, Moonshot, Qwen, vLLM, Ollama/v1, …) by configuringbase_url+api_key+modelin TOML. The cloudopenai.pyis untouched — mirrors how the embeddings registry already splitsollama/mistral/jinainto separate providers instead of overloadingopenaiwith abase_urlswitch.Why
provider = "openai"was overloaded (SDK protocol vs. cloud), so wizard hints andDEFAULT_TEXT_MODELSwere misleading for custom endpoints.DEFAULT_TEXT_MODELS["openai"] = "gpt-4o-mini"is wrong for vLLM/Ollama/DeepSeek — compatible endpoints have no sensible default model.Changes
common/llm/openai_compatible.pyOpenAICompatibleLlm:base_urlrequired,api_keyoptional,modelper-call; inlines_resolve_secret_ref(env:/file:, avoidscommon→enginecycle,TODO(secrets))common/llm/__init__.pyget_provider(name, **kwargs)forwards kwargsconfig.py[summary]/[description]gainbase_url/api_key(default"", read only for this provider)common/summary.py,common/vlm.py_ensure_llmforwards endpoint configserver/setup_wizard.py_prompt_openai_compatible_endpoint()(base_url/api_key/model) +_applywrites the fieldscommon/llm/anthropic.py,common/llm/gemini.py__init__accepts**_kwargs(tolerate uniform forwarding)tests/test_llm_registry.py,tests/test_setup_wizard_sections.py_applycasesdocs/providers.md,docs/configuration.mdOPENAI_API_KEYnoteKey decisions
api_key. Absentapi_keywould makeAsyncOpenAIauto-readOPENAI_API_KEY— silently pointing a custom endpoint at OpenAI cloud creds. A placeholder is substituted instead (locked by a test).CredentialService.resolve— it lives inengine, a higher layer; lifting it tocommon.secretsis a separate refactor, tracked viaTODO(secrets).Backward compatibility
openai.pyunchanged (env path preserved); new fields default empty;get_provider's added**kwargsis signature-compatible. Caveat:base_url/api_keyare not in the transformation-cache key — a switch without amodelchange would return cached summaries from the prior endpoint (low risk in practice; a switch is always accompanied by amodelchange).Example