Update documentation about accelerators #2943
Open
+45
−82
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dtrawins
changed the title
ngrozae
reviewed
Jan 7, 2025
docs/accelerators.md
Outdated
|
|
||
| 1. Use a .json configuration file to set the `--target_device` parameter with the pattern of: `MULTI:<DEVICE_1>,<DEVICE_2>`. | ||
| The order of the devices will define their priority, in this case making `device_1` the primary selection. | ||
| OpenVINO Model Server can support using [NPU device](https://docs.openvino.ai/canonical/openvino_docs_install_guides_configurations_for_intel_npu.html) |
There was a problem hiding this comment.
ngrozae
reviewed
Jan 7, 2025
docs/accelerators.md
Outdated
Comment on lines
56
to
57
| Docker image with required dependencies can be build using this procedure: | ||
| The docker image of OpenVINO Model Server including support for NVIDIA can be built from sources |
There was a problem hiding this comment.
now it uses one line
Suggested change
| Docker image with required dependencies can be build using this procedure: | |
| The docker image of OpenVINO Model Server including support for NVIDIA can be built from sources | |
| Docker image with required dependencies can be build using this procedure: | |
| The docker image of OpenVINO Model Server including support for NVIDIA can be built from sources |
dkalinowski
reviewed
Jan 8, 2025
| --model_path /opt/model --model_name resnet --port 9001 \ | ||
| --plugin_config '{"AUTO_BATCH_TIMEOUT": 200}' \ | ||
| --target_device BATCH:CPU(16) |
There was a problem hiding this comment.
You don't specify what is the (16) in this example - maybe its worth mentioning? Do I understand it correct that it is max batch size when grouping together?
mzegla
reviewed
Jan 13, 2025
| char bracket = '('; | ||
| auto bracketPos = deviceName.find(bracket); | ||
| if (bracketPos != std::string::npos) { | ||
| deviceName = deviceName.substr(0, bracketPos); |
There was a problem hiding this comment.
Why are we doing this? Don't we exclude batching info here?
There was a problem hiding this comment.
to get physical device and retrieve supported properties, we need to omit content in brackets. For the target device in model compilation, it should be included.
Co-authored-by: Miłosz Żeglarski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🛠 Summary
CVS-149070
Drop NVIDIA plugin from the maintained list
Drop MULTI plugin - replaced by AUTO with THROUGHPUT_CUMULATIVE hint
Add BATCH virtual device
Refresh documentation about accelerators
🧪 Checklist
``