Skip to content

CVS-174235 : OV SDK Version Check Removed for EpCtx Consumption Flow on NPU #837

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ankitm3k merged 2 commits into from
Jan 8, 2026
Merged

CVS-174235 : OV SDK Version Check Removed for EpCtx Consumption Flow on NPU #837

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9a217ef
Disable EPCtx Model OV SDK Version Check for NPU only
ankitm3k Oct 1, 2025
e51c5e0
fix: pass device_type as string
ankitm3k Jan 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions onnxruntime/core/providers/openvino/backend_manager.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,11 +100,10 @@
std::string exception_str =
"[OpenVINO-EP] Bounded dynamic model execution using provider option reshape_input is not supported for OVEP EPContext model";
ORT_THROW(exception_str);
}
if (subgraph_context_.is_ep_ctx_ovir_encapsulated) {
model_stream = ep_ctx_handle_.GetModelBlobStream(session_context_.onnx_model_path_name.replace_extension("xml").string(), subgraph);
} if (subgraph_context_.is_ep_ctx_ovir_encapsulated) {

Check notice on line 103 in onnxruntime/core/providers/openvino/backend_manager.cc

View workflow job for this annotation

GitHub Actions / cpplint

[cpplint] onnxruntime/core/providers/openvino/backend_manager.cc#L103 

 Did you mean "else if"? If not, start a new line for "if".  [readability/braces] [4]
Raw output 
onnxruntime/core/providers/openvino/backend_manager.cc:103:  Did you mean "else if"? If not, start a new line for "if".  [readability/braces] [4]
model_stream = ep_ctx_handle_.GetModelBlobStream(session_context_.onnx_model_path_name.replace_extension("xml").string(), subgraph, session_context_.device_type);
} else {
model_stream = ep_ctx_handle_.GetModelBlobStream(session_context_.so_context_file_path, subgraph);
model_stream = ep_ctx_handle_.GetModelBlobStream(session_context_.so_context_file_path, subgraph, session_context_.device_type);
}

} else {
Expand Down
12 changes: 7 additions & 5 deletions onnxruntime/core/providers/openvino/onnx_ctx_model_helper.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ Status EPCtxHandler::AddOVEPCtxNodeToGraph(const GraphViewer& graph_viewer,
return Status::OK();
}

std::unique_ptr<ModelBlobWrapper> EPCtxHandler::GetModelBlobStream(const std::filesystem::path& so_context_file_path, const GraphViewer& graph_viewer) const {
std::unique_ptr<ModelBlobWrapper> EPCtxHandler::GetModelBlobStream(const std::filesystem::path& so_context_file_path, const GraphViewer& graph_viewer, const std::string& device_type) const {
auto first_index = *graph_viewer.GetNodesInTopologicalOrder().begin();
auto node = graph_viewer.GetNode(first_index);
ORT_ENFORCE(node != nullptr);
Expand Down Expand Up @@ -128,10 +128,12 @@ std::unique_ptr<ModelBlobWrapper> EPCtxHandler::GetModelBlobStream(const std::fi
// If the model stream is not an XML (i.e. precompiled blob), the OpenVINO SDK version that it was
// exported with must match the version that is currently running.
native_blob_path = std::move(blob_filepath);
ORT_ENFORCE((attrs.count(EP_SDK_VER) == 1) && (attrs.at(EP_SDK_VER).s() == openvino_sdk_version_),
Copy link

@MayureshV1 MayureshV1 Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we specifically remove this check for NPU device only for now?

Copy link
Author

@ankitm3k ankitm3k Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed as requested

Copy link

@MayureshV1 MayureshV1 Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ankitm3k . Functionally the change looks good and meets the requirement.

Unfortunately, the side effect of this device specific behavior is to pass the session context to ctx_helper. Is there any way to avoid. Should we just pass device_type?
@javier-intel , @ericcraw, @preetha-intel .. any suggestions?

Copy link
Author

@ankitm3k ankitm3k Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have amended the change to accept std::string device_type to keep things simple & clean

"EPCtx blob was exported / is compatible with OpenVINO SDK version " + attrs.at(EP_SDK_VER).s() +
", but OpenVINO SDK version currently in use is " + openvino_sdk_version_);

// Skip SDK version check for NPU devices as they may use different SDK versions.
if (device_type.find("NPU") == std::string::npos) {
ORT_ENFORCE((attrs.count(EP_SDK_VER) == 1) && (attrs.at(EP_SDK_VER).s() == openvino_sdk_version_),
"EPCtx blob was exported / is compatible with OpenVINO SDK version " + attrs.at(EP_SDK_VER).s() +
", but OpenVINO SDK version currently in use is " + openvino_sdk_version_);
}
result.reset(); // Release the stream as we will get the native blob from SharedContext
auto shared_context = shared_context_manager_->GetOrCreateSharedContext(native_blob_path);
return std::make_unique<ModelBlobWrapper>(shared_context->GetNativeBlobAsStream(partition_name), shared_context->GetNativeBlob(partition_name));
Expand Down
2 changes: 1 addition & 1 deletion onnxruntime/core/providers/openvino/onnx_ctx_model_helper.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class EPCtxHandler {
const std::string& graph_name,
const bool embed_mode,
std::string&& model_blob_str) const;
std::unique_ptr<ModelBlobWrapper> GetModelBlobStream(const std::filesystem::path& so_context_file_path, const GraphViewer& subgraph_view) const;
std::unique_ptr<ModelBlobWrapper> GetModelBlobStream(const std::filesystem::path& so_context_file_path, const GraphViewer& subgraph_view, const std::string& device_type) const;
InlinedVector<const Node*> GetEPCtxNodes() const;
bool CheckEPCacheContextAttribute(const GraphViewer& subgraph_view, const std::string& target_attr_extn) const;
std::shared_ptr<SharedContext> Initialize(const std::vector<IExecutionProvider::FusedNodeAndGraph>& fused_nodes, const SessionContext& session_context);
Expand Down
Loading