Skip to content

Commit 6caf21f

Browse files
authored
Add OCI LangChain support for hosted Nemotron workflows (#1804)
## Summary - add a first-class OCI LLM config to NAT core and register it alongside the existing providers - add LangChain wrapper support for OCI via `langchain-oci`, matching the workflow-layer integration shape used by AWS Bedrock in this repo - add OCI docs and tests, with live integration coverage centered on an OCI-hosted Nemotron inference endpoint - declare the necessary `uv` extra conflicts so the workspace remains solvable when `langchain-oci` introduces `openai>=2` alongside existing `strands` and `vanna` surfaces ## What Was Tested - `PYTHONPATH=$(pwd)/packages/nvidia_nat_core/src:$(pwd)/packages/nvidia_nat_langchain/src:$(pwd)/packages/nvidia_nat_test/src .venv/bin/pytest packages/nvidia_nat_core/tests/nat/llm/test_oci_llm.py packages/nvidia_nat_langchain/tests/test_llm_langchain.py -q -k 'OCI or oci'` - `OCI_NEMOTRON_BASE_URL=http://127.0.0.1:8080/v1 OCI_NEMOTRON_MODEL=nvidia/Llama-3.1-Nemotron-Nano-8B-v1 PYTHONPATH=$(pwd)/packages/nvidia_nat_core/src:$(pwd)/packages/nvidia_nat_langchain/src:$(pwd)/packages/nvidia_nat_test/src .venv/bin/pytest packages/nvidia_nat_langchain/tests/test_langchain_agents.py -q --run_integration -k oci_hosted_nemotron_openai_compatible_agent` - `uv lock` ## Notes - all live validation in this PR is centered on `nvidia/Llama-3.1-Nemotron-Nano-8B-v1` - the live Nemotron endpoint is served from an OKE + vLLM inference layer in Phoenix - this closes the main OCI workflow-layer gap relative to the existing AWS Bedrock path in `nvidia_nat_langchain` ## Summary by CodeRabbit * **New Features** * Added an OCI-hosted LLM provider with LangChain client support and Nemotron-compatible model option. * **Documentation** * New OCI Generative AI integration guide, config examples, TOC entry, and documentation redirects; OCI added to supported providers list. * **Tests** * Added unit and integration tests for the OCI provider and LangChain wrapper, plus a pytest fixture for OCI Nemotron endpoints. * **Chores** * Broadened OpenAI dependency ranges, added langchain-oci runtime dependency, and adjusted optional extras/conflicts in packaging. Authors: - Federico Kamelhar (https://github.com/fede-kamel) - Will Killian (https://github.com/willkill07) Approvers: - Will Killian (https://github.com/willkill07) - https://github.com/Salonijain27 URL: #1804
1 parent ca59672 commit 6caf21f

56 files changed

Lines changed: 3879 additions & 7 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

ci/markdown-link-check-config.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@
2323
},
2424
{
2525
"pattern": "^https://docs\\.cursor\\.com"
26+
},
27+
{
28+
"pattern": "^https://docs\\.docker\\.com"
2629
}
2730
]
2831
}

docs/source/build-workflows/llms/index.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ NVIDIA NeMo Agent Toolkit supports the following LLM providers:
2828
| [OpenAI](https://openai.com) | `openai` | OpenAI API |
2929
| [AWS Bedrock](https://aws.amazon.com/bedrock/) | `aws_bedrock` | AWS Bedrock API |
3030
| [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/quickstart) | `azure_openai` | Azure OpenAI API |
31+
| [OCI Generative AI](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm) | `oci` | OCI Generative AI |
3132
| [LiteLLM](https://github.com/BerriAI/litellm) | `litellm` | LiteLLM API |
3233
| [Hugging Face](https://huggingface.co) | `huggingface` | Hugging Face API |
3334
| [Hugging Face Inference](https://huggingface.co/docs/api-inference) | `huggingface_inference` | Hugging Face Inference API, Endpoints, and TGI |
@@ -52,6 +53,15 @@ llms:
5253
azure_openai_llm:
5354
_type: azure_openai
5455
azure_deployment: gpt-4o-mini
56+
oci_llm:
57+
_type: oci
58+
model_name: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
59+
region: us-chicago-1
60+
compartment_id: ocid1.compartment.oc1..example
61+
auth_type: API_KEY
62+
auth_profile: DEFAULT
63+
auth_file_location: ~/.oci/config
64+
provider: meta
5565
litellm_llm:
5666
_type: litellm
5767
model_name: gpt-4o
@@ -118,6 +128,39 @@ The AWS Bedrock LLM provider is defined by the {py:class}`~nat.llm.aws_bedrock_l
118128
* `credentials_profile_name` - The credentials profile name to use for the model
119129
* `max_retries` - The maximum number of retries for the request
120130

131+
### OCI Generative AI
132+
133+
You can use the following fields to configure the OCI Generative AI LLM provider:
134+
135+
* `region` - OCI region for the Generative AI service (defaults to `us-chicago-1`). The service endpoint is derived automatically.
136+
* `endpoint` - Optional explicit endpoint URL. Overrides the region-derived endpoint when set.
137+
* `compartment_id` - The OCI compartment OCID used for inference requests
138+
* `auth_type` - OCI SDK auth mode such as `API_KEY`, `SECURITY_TOKEN`, `INSTANCE_PRINCIPAL`, or `RESOURCE_PRINCIPAL`
139+
* `auth_profile` - OCI config profile name for file-backed auth
140+
* `auth_file_location` - Path to the OCI config file
141+
* `provider` - Optional provider override such as `meta`, `google`, `cohere`, or `openai`
142+
143+
The OCI Generative AI LLM provider is defined by the {py:class}`~nat.llm.oci_llm.OCIModelConfig` class.
144+
145+
* `model_name` - The name of the model to use
146+
* `region` - OCI region (defaults to `us-chicago-1`). The endpoint is derived from `https://inference.generativeai.{region}.oci.oraclecloud.com`.
147+
* `endpoint` - Optional explicit endpoint URL. Overrides the region-derived endpoint.
148+
* `compartment_id` - OCI compartment OCID
149+
* `auth_type` - OCI SDK auth type
150+
* `auth_profile` - OCI profile name for file-backed auth
151+
* `auth_file_location` - Path to the OCI config file
152+
* `provider` - Optional OCI provider override such as `meta`, `google`, `cohere`, or `openai`
153+
* `temperature` - The temperature to use for the model
154+
* `top_p` - The top-p value to use for the model
155+
* `max_tokens` - The maximum number of tokens to generate
156+
* `seed` - The seed to use for the model
157+
* `max_retries` - The maximum number of retries for the request
158+
* `request_timeout` - HTTP request timeout in seconds
159+
160+
:::{note}
161+
This provider targets OCI Generative AI through the OCI SDK-backed `langchain-oci` path and does not enable the Responses API.
162+
:::
163+
121164
### Azure OpenAI
122165

123166
You can use the following environment variables to configure the Azure OpenAI LLM provider:

docs/source/components/integrations/index.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,5 @@ limitations under the License.
2323
./frameworks.md
2424
./a2a.md
2525
AWS Bedrock <./integrating-aws-bedrock-models.md>
26-
```
26+
OCI Generative AI <./integrating-oci-generative-ai-models.md>
27+
```
Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
<!--
2+
SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
SPDX-License-Identifier: Apache-2.0
4+
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
-->
17+
18+
# NVIDIA NeMo Agent Toolkit OCI Integration
19+
20+
The NeMo Agent Toolkit supports integration with multiple [LLM](../../build-workflows/llms/index.md) providers, including OCI Generative AI. The `oci` provider uses OCI SDK authentication and is designed for OCI Generative AI model and endpoint access. For workflow parity with the AWS Bedrock path, the toolkit also includes a LangChain wrapper built on `langchain-oci`.
21+
22+
To view the full list of supported LLM providers, run `nat info components -t llm_provider`.
23+
24+
## Configuration
25+
26+
### Prerequisites
27+
Before integrating OCI, ensure you have:
28+
29+
- access to OCI Generative AI in the target region
30+
- a valid OCI auth method such as `API_KEY`, `SECURITY_TOKEN`, `INSTANCE_PRINCIPAL`, or `RESOURCE_PRINCIPAL`
31+
- the target compartment OCID
32+
- the target OCI region (defaults to `us-chicago-1`) or a custom endpoint URL
33+
34+
Common deployment patterns include:
35+
36+
- OCI Generative AI regional endpoints
37+
- custom OCI Generative AI endpoints
38+
- OCI-hosted inference for NVIDIA Nemotron used as a live integration target
39+
40+
### Example Configuration
41+
Add the OCI LLM configuration to your workflow config file:
42+
43+
```yaml
44+
llms:
45+
oci_llm:
46+
_type: oci
47+
model_name: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
48+
region: us-chicago-1
49+
compartment_id: ocid1.compartment.oc1..example
50+
auth_type: API_KEY
51+
auth_profile: DEFAULT
52+
temperature: 0.0
53+
max_tokens: 1024
54+
top_p: 1.0
55+
request_timeout: 60
56+
```
57+
58+
### Configurable Options
59+
* `model_name`: The name of the OCI-hosted model to use (required)
60+
* `region`: OCI region for the Generative AI service (defaults to `us-chicago-1`). The service endpoint is derived automatically.
61+
* `endpoint`: Optional explicit service endpoint URL. Overrides the region-derived endpoint when set.
62+
* `compartment_id`: OCI compartment OCID
63+
* `auth_type`: OCI SDK auth type
64+
* `auth_profile`: OCI profile name for file-backed auth
65+
* `auth_file_location`: Path to the OCI config file
66+
* `provider`: Optional OCI provider override such as `meta`, `google`, `cohere`, or `openai`
67+
* `temperature`: Controls randomness in the output (0.0 to 1.0)
68+
* `max_tokens`: Maximum number of tokens to generate
69+
* `top_p`: Top-p sampling parameter (0.0 to 1.0)
70+
* `seed`: Optional random seed
71+
* `max_retries`: Maximum number of retries for the request
72+
* `request_timeout`: HTTP request timeout in seconds
73+
74+
### Limitations
75+
* This provider targets OCI Generative AI through the OCI SDK-backed `langchain-oci` path.
76+
* The Responses API is not enabled for this provider in the current release.
77+
78+
## Nemotron On OCI
79+
80+
One strong OCI deployment pattern is NVIDIA Nemotron hosted on OCI and exposed through an OpenAI-compatible route. In that setup, the toolkit can validate live integration behavior against the OCI-hosted Nemotron endpoint while the official provider and LangChain wrapper cover the OCI Generative AI path.
81+
82+
## Usage
83+
Reference the OCI LLM in your configuration:
84+
85+
```yaml
86+
llms:
87+
oci_llm:
88+
_type: oci
89+
model_name: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
90+
region: us-chicago-1
91+
compartment_id: ocid1.compartment.oc1..example
92+
auth_profile: DEFAULT
93+
```
94+
95+
## Troubleshooting
96+
* `401 Unauthorized`: verify the OCI profile, signer, and IAM permissions for Generative AI.
97+
* `404 Not Found`: confirm the regional endpoint or custom endpoint URL is correct.
98+
* `Connection errors`: verify OCI networking and that the regional endpoint is reachable.
99+
* `Tool calling issues`: verify the served model supports tool calling and that the serving stack is configured for it.

docs/source/get-started/installation.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ The following [LLM](../build-workflows/llms/index.md) API providers are supporte
2727
- OpenAI
2828
- AWS Bedrock
2929
- Azure OpenAI
30+
- OCI Generative AI
3031

3132
## Packages
3233

examples/A2A/math_assistant_a2a/uv.lock

Lines changed: 77 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)