Agent: add support for remote server #1644

alexsin368 · 2025-05-02T00:42:38Z

Description

Add support for usage of a remote server inference endpoint. Supports Enterprise Inference.

Related PR: opea-project/GenAIExamples#1900

Issues

N/A

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)
Others (enhancement, documentation, validation, etc.)

Dependencies

None

Tests

Tested with AgentQnA consuming this agent microservice where chat completions were showing on the UI.

Signed-off-by: alexsin368 <[email protected]>

for more information, see https://pre-commit.ci

comps/agent/src/integrations/utils.py

comps/agent/src/integrations/config.py

Signed-off-by: alexsin368 <[email protected]>

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <[email protected]>

for more information, see https://pre-commit.ci

alexsin368

removed use_remote_service

Copilot

Pull Request Overview

This PR adds support for using a remote server inference endpoint by enabling the use of an API key when initializing the ChatOpenAI client. Key changes include:

Updating the ChatOpenAI initialization in utils.py to append "/v1" to the remote endpoint when an API key is provided.
Adding a new argparse argument "--api_key" and corresponding environment variable handling in config.py.
Updating README.md to document how to use remote servers with the new API key support.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
comps/agent/src/integrations/utils.py	Adds conditional initialization of ChatOpenAI using an API key.
comps/agent/src/integrations/config.py	Supports reading the api_key from environment variables.
comps/agent/src/README.md	Updates documentation to instruct on running LLMs with remote servers.

Comments suppressed due to low confidence (1)

comps/agent/src/README.md:119

[nitpick] The documentation uses 'LLM_ENDPOINT_URL' while the configuration code expects 'llm_endpoint_url'. Consider aligning the environment variable naming for consistency.

- `LLM_ENDPOINT_URL`=<inference-endpoint>

yinghu5

LGTM, thank you!

comps/agent/src/integrations/utils.py

Signed-off-by: alexsin368 <[email protected]>

comps/agent/src/integrations/utils.py

Signed-off-by: alexsin368 <[email protected]>

for more information, see https://pre-commit.ci

This reverts commit b7c4acf.

Signed-off-by: alexsin368 <[email protected]>

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: ZePan110 <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: Ye, Xinyu <[email protected]> Co-authored-by: Lv,Liang1 <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: ZePan110 <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: alexsin368 <[email protected]>

for more information, see https://pre-commit.ci Signed-off-by: alexsin368 <[email protected]>

…pea-project#1643) * bug fix for dataingest url Signed-off-by: Mustafa <[email protected]> * add validation function Signed-off-by: Mustafa <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * validation update Signed-off-by: Mustafa <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update validation function Signed-off-by: Mustafa <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mustafa <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: alexsin368 <[email protected]>

…roject#1645) * Add MariaDB Vector third-party service MariaDB Vector was introduced since MariaDB Server 11.7 Signed-off-by: Razvan-Liviu Varzaru <[email protected]> * Add retriever MariaDB Vector integration Signed-off-by: Razvan-Liviu Varzaru <[email protected]> * Add dataprep MariaDB Vector integration Signed-off-by: Razvan-Liviu Varzaru <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix CI failures - md5 is used for the primary key not as a security hash - fixed mariadb readme headers Signed-off-by: Razvan-Liviu Varzaru <[email protected]> --------- Signed-off-by: Razvan-Liviu Varzaru <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: chensuyue <[email protected]> Signed-off-by: alexsin368 <[email protected]>

…oject#1676) * Expand test matrix, find all tests use 3rd party Dockerfiles Signed-off-by: chensuyue <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Update README.md for first entry of OPEA Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

…#1635) Signed-off-by: alexsin368 <[email protected]>

* Fix huggingfacehub API upgrade issue Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

…pea-project#1694) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: alexsin368 <[email protected]>

This reverts commit b7c4acf. Signed-off-by: alexsin368 <[email protected]>

Signed-off-by: alexsin368 <[email protected]>

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <[email protected]>

for more information, see https://pre-commit.ci

alexsin368 · 2025-05-15T01:42:22Z

need help resolving DCO. steps i followed are not working

louie-tsai

looks good to me

alexsin368 · 2025-05-15T22:00:44Z

ready to merge!

alexsin368 added 2 commits April 30, 2025 17:56

add support for remote server

369d5dc

Signed-off-by: alexsin368 <[email protected]>

add steps to enable remote server

0f6191d

Signed-off-by: alexsin368 <[email protected]>

alexsin368 requested review from lkk12014402 and minmin-intel as code owners May 2, 2025 00:42

[pre-commit.ci] auto fixes from pre-commit.com hooks

71f1608

for more information, see https://pre-commit.ci

alexsin368 mentioned this pull request May 2, 2025

AgentQnA - add support for remote server opea-project/GenAIExamples#1900

Merged

4 tasks

louie-tsai reviewed May 2, 2025

View reviewed changes

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

comps/agent/src/integrations/config.py Outdated Show resolved Hide resolved

alexsin368 and others added 3 commits May 2, 2025 16:58

remove use_remote_service

bbcda06

Signed-off-by: alexsin368 <[email protected]>

Merge branch 'agent-remote-service' of https://github.com/alexsin368/…

03f81c6

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

45cf931

for more information, see https://pre-commit.ci

alexsin368 commented May 3, 2025

View reviewed changes

yinghu5 requested review from xuechendi, yinghu5 and Copilot May 7, 2025 08:35

Copilot AI reviewed May 7, 2025

View reviewed changes

Merge branch 'main' into agent-remote-service

35c67df

yinghu5 requested a review from ftian1 as a code owner May 12, 2025 01:49

yinghu5 approved these changes May 12, 2025

View reviewed changes

Merge branch 'main' into agent-remote-service

836c0dd

minmin-intel reviewed May 13, 2025

View reviewed changes

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

add OpenAI models instructions, fix format of commands

4899f79

Signed-off-by: alexsin368 <[email protected]>

louie-tsai reviewed May 14, 2025

View reviewed changes

comps/agent/src/integrations/utils.py Outdated Show resolved Hide resolved

minmin-intel approved these changes May 15, 2025

View reviewed changes

alexsin368 and others added 7 commits May 14, 2025 17:34

Merge branch 'main' into agent-remote-service

101d133

simplify ChatOpenAI instantiation

b7c4acf

Signed-off-by: alexsin368 <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

6586657

for more information, see https://pre-commit.ci

Revert "simplify ChatOpenAI instantiation"

d288734

This reverts commit b7c4acf.

add back check and logic for llm_engine, set openai_key argument

848368f

Signed-off-by: alexsin368 <[email protected]>

Merge branch 'agent-remote-service' of https://github.com/alexsin368/…

7d01d77

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

53aaaa5

for more information, see https://pre-commit.ci

ZePan110 and others added 24 commits May 14, 2025 18:34

Provide ARCH option for lvm-video-llama image build (opea-project#1630)

a70201f

Signed-off-by: ZePan110 <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Add sglang microservice for supporting llama4 model (opea-project#1640)

212e612

Signed-off-by: Ye, Xinyu <[email protected]> Co-authored-by: Lv,Liang1 <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Remove invalid codeowner. (opea-project#1642)

5fc478e

Signed-off-by: ZePan110 <[email protected]> Signed-off-by: alexsin368 <[email protected]>

add support for remote server

1fe684c

Signed-off-by: alexsin368 <[email protected]>

add steps to enable remote server

bd68f54

Signed-off-by: alexsin368 <[email protected]>

remove use_remote_service

23f1f56

Signed-off-by: alexsin368 <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

d1d2ac1

for more information, see https://pre-commit.ci Signed-off-by: alexsin368 <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

a9d9ad7

for more information, see https://pre-commit.ci Signed-off-by: alexsin368 <[email protected]>

update PR reviewers (opea-project#1651)

5e1656f

Signed-off-by: chensuyue <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Expand test matrix, find all tests use 3rd party Dockerfiles (opea-pr…

69fea0d

…oject#1676) * Expand test matrix, find all tests use 3rd party Dockerfiles Signed-off-by: chensuyue <[email protected]> Signed-off-by: alexsin368 <[email protected]>

fix the typo of README.md Comp (opea-project#1679)

388c264

Update README.md for first entry of OPEA Signed-off-by: alexsin368 <[email protected]>

Fix request handle timeout issue (opea-project#1687)

3b42858

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

FEAT: Enable OPEA microservices to start as MCP servers (opea-project…

928e0f7

…#1635) Signed-off-by: alexsin368 <[email protected]>

Fix huggingface_hub API upgrade issue (opea-project#1691)

9be8f9f

* Fix huggingfacehub API upgrade issue Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

add OpenAI models instructions, fix format of commands

0ffa6a6

Signed-off-by: alexsin368 <[email protected]>

Fix dataprep opensearch ingest issue (opea-project#1697)

f83070c

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

Fix embedding issue with ArangoDB due to deprecated HuggingFace API (o…

72bc23b

…pea-project#1694) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: alexsin368 <[email protected]>

simplify ChatOpenAI instantiation

b2d93ff

Signed-off-by: alexsin368 <[email protected]>

Revert "simplify ChatOpenAI instantiation"

78001b0

This reverts commit b7c4acf. Signed-off-by: alexsin368 <[email protected]>

add back check and logic for llm_engine, set openai_key argument

1f4b746

Signed-off-by: alexsin368 <[email protected]>

Merge branch 'agent-remote-service' of https://github.com/alexsin368/…

79e0407

…GenAIComps into agent-remote-service Signed-off-by: alexsin368 <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

45376b9

for more information, see https://pre-commit.ci

louie-tsai approved these changes May 15, 2025

View reviewed changes

yinghu5 merged commit fcf7a5a into opea-project:main May 16, 2025
13 checks passed

alexsin368 deleted the agent-remote-service branch May 16, 2025 23:37

joshuayao mentioned this pull request May 19, 2025

[Feature] Enable remote inference endpoints for examples opea-project/GenAIExamples#1973

Open

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Agent: add support for remote server #1644

Agent: add support for remote server #1644

Uh oh!

alexsin368 commented May 2, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexsin368 left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

yinghu5 left a comment

Uh oh!

Uh oh!

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

louie-tsai left a comment

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

Uh oh!

Uh oh!

Agent: add support for remote server #1644

Agent: add support for remote server #1644

Uh oh!

Conversation

alexsin368 commented May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues

Type of change

Dependencies

Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexsin368 left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

yinghu5 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

louie-tsai left a comment

Choose a reason for hiding this comment

Uh oh!

alexsin368 commented May 15, 2025

Uh oh!

Uh oh!

Uh oh!

alexsin368 commented May 2, 2025 •

edited

Loading