feat(providers): add NeuralWatt energy-efficient inference provider#242
feat(providers): add NeuralWatt energy-efficient inference provider#242gr3enarr0w wants to merge 7 commits into
Conversation
NeuralWatt (api.neuralwatt.com/v1) is a direct single-vendor OpenAI-compatible inference service hosting open-source models (Llama-3.3-70B, GLM-5.1-FP8, etc.) with per-request energy transparency metrics. It is a direct provider, not an aggregator — no cost opacity concern. Implements Chat, Stream, Discovery, and Proxy capabilities. The /v1/models endpoint is publicly accessible without authentication. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…g config entries - Fix SupportsModel() returning true unconditionally (dead loop bug) — now returns false for models not in the static list, preventing NeuralWatt from claiming gpt-4o, claude-3-opus, etc. and mis-routing those requests - Hoist SupportedModels slice to package-level var to avoid per-call allocation - Add Authorization: Bearer header assertion to Complete and CompleteStream tests - Add TestNeuralWattProvider_DiscoverModels_MockHTTP test for CapabilityDiscovery - Add neuralwatt entry to config.example.json, config.example.yaml, docker-compose.yml per AGENTS.md steps 5-6 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
| @@ -0,0 +1,16 @@ | |||
| #!/usr/bin/env bash | |||
| set -euo pipefail | |||
There was a problem hiding this comment.
Can you remove .superset folder?
|
Nice and clean, thanks. Pulled the branch, it builds and the neuralwatt and stability tests pass locally, and it follows the OpenAI compatible pattern well (shared PostChat and PostStream, the discovery helper, and all four interface assertions). Two things before merge:
On the catalog side: since NeuralWatt exposes pricing and context windows in its models feed, I drafted the model-catalog entries in ferro-labs/model-catalog#25 so these models are priced for cost optimized routing and the budget plugin. Feel free to take that over or adjust it. For future providers the cleaner flow is to raise the model-catalog PR yourself alongside the gateway PR. And to close the loop on the earlier question: NeuralWatt is a direct host, not an aggregator, so this is on the normal track and does not need to wait behind Requesty or NanoGPT. |
…ling scaffold The hardcoded SupportedModels seed listed models NeuralWatt no longer serves (meta-llama/Llama-3.3-70B-Instruct, zai-org/GLM-5.1-FP8, meta-llama/Llama-3.1-8B-Instruct). With live discovery off, SupportsModel matched nothing NeuralWatt actually returns. Replaced with the current live /v1/models list: qwen3.5-397b(-fast), qwen3.6-35b(-fast), kimi-k2.6(-fast), kimi-k2.7-code. Also drops .superset/ and CODEX.md (unrelated local dev-tooling scaffold, not part of this provider). Note: TestCatalogBackupCoversRegisteredProviders still fails in CI - the embedded catalog_backup.json is a generated mirror of ferro-labs/model-catalog releases and has no neuralwatt entries yet. That's blocked on ferro-labs/model-catalog#25 merging and a catalog refresh, not fixable from this repo.
Also finishes staging the neuralwatt seed-model fix from the previous commit (the earlier commit only picked up the .superset/CODEX.md deletions due to a bad git add pathspec).
|
Warning Review limit reached
Next review available in: 59 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Fixed the stale seed model list per your review — replaced the three dead IDs with the current live That test still won't go green from this side, though: Also dropped One more thing: the CI run right after I pushed showed |
2 participants
Closes #245
Summary
Adds NeuralWatt (
api.neuralwatt.com) as a new direct inference provider. NeuralWatt hosts open-source models on energy-efficient hardware with a fully OpenAI-compatible API.Provider details
https://api.neuralwatt.com/v1Authorization: Bearer $NEURALWATT_API_KEY/v1/chat/completions,/v1/models)chat,stream,proxy,discoverymeta-llama/Llama-3.3-70B-Instruct,zai-org/GLM-5.1-FP8,meta-llama/Llama-3.1-8B-InstructChanges
providers/neuralwatt/neuralwatt.go— implementation usingopenaicompat.PostChat/PostStreamanddiscov.DiscoverOpenAICompatibleModelsproviders/neuralwatt/neuralwatt_test.go— auth header assertions,DiscoverModelstestproviders/names.go—NameNeuralWattproviders/providers_list.go—ProviderEntryregistrationproviders/stability_test.go— stability caseconfig.example.json,config.example.yaml—neuralwatttargetdocker-compose.yml—NEURALWATT_API_KEYcommentTest plan
go test ./providers/neuralwatt/...go test ./providers/ -run TestProviderNameStabilitygo test ./...🤖 Generated with Claude Code