Problem Statement
Currently the open router host name is hardcoded. By making it configurable, it would be possible to use any OpenAI compatible API host, incl. local inference via e.g. LM Studio.
Proposed Solution
It should be sufficient to allow configuring of the URL the same way as the API key is configured currently.
Alternatives Considered
No response
Feature Area
Policy engine / evaluation
Related Harness
None
Use Case
If the purpose of a policy is to protect from leaking sensitive data to models, there needs to be a way for running LLM as a judge with local inference. Sending data to a remote host defeats this purpose.
Additional Context
No response
Problem Statement
Currently the open router host name is hardcoded. By making it configurable, it would be possible to use any OpenAI compatible API host, incl. local inference via e.g. LM Studio.
Proposed Solution
It should be sufficient to allow configuring of the URL the same way as the API key is configured currently.
Alternatives Considered
No response
Feature Area
Policy engine / evaluation
Related Harness
None
Use Case
If the purpose of a policy is to protect from leaking sensitive data to models, there needs to be a way for running LLM as a judge with local inference. Sending data to a remote host defeats this purpose.
Additional Context
No response