Integrate Automated QDQ placement tool - Part 3

[willg-nv]

What does this PR do?

Type of change: new feature

Overview: This PR integrates automated QDQ placement tool to ModelOpt. This PR is 3/4 of the change. This PR contains the following changes:

1. Implements QDQAutotuner and Autotuner CLI interface
2. Implements Benchmark to measure E2E time of QDQ models.
3. unit tests for QDQ Autotuner and config.

Part 1: #701
Part 2: #702
Part 3: #703
Part 4: #704

Usage

python -m modelopt.onnx.quantization.autotune --model model.onnx

Testing

Implemented unit tests for QDQAutotuner and Config classes.

Before your PR is "Ready for review"

• Make sure you read and follow Contributor guidelines and your commits are signed.
• Is this change backward compatible?: Yes
• Did you write any new necessary tests?: Yes
• Did you add or update any necessary documentation?: No, document will be in part 4.
• Did you update Changelog?: No. change log will be in part 4.

Additional Information

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[willg-nv]

@vishalpandya1990 could you help me review this PR? thanks!

[vishalpandya1990]

@vishalpandya1990 could you help me review this PR? thanks!

Sorry for the delay. Added Ajinkya for review.

[gcunhase]

Can we update this behavior to match the ONNX quantization and Autocast workflow?

In there, if output_path is not given, the resulting model is saved in the same path as the input model with a name extension. For ex: the quantized model.onnx is saved as model.quant.onnx and the converted model is saved as model.fp16.onnx.

See more details in:

Model-Optimizer/modelopt/onnx/quantization/quantize.py

Line 391 in 6f18490

if not output_path:

[gcunhase]

Suggestion: rename this as output_path to match other ONNX workflows.

[willg-nv]

FYI: I plan to create another PR to integrate autotuner as a sub-command of modelopt.onnx.quantization. User could 1) directly run autotuner, 2) or autotune based on PTQ model. After that, I think cli.py could be removed.

[gcunhase]

Can we move this to tests/_test_utils/onnx/lib_test_models.py? Alternatively, NonSimplifiedModel or build_resnet_block could be used here instead?

@ajrasane WDYT?