Skip to content

Using Validation Framework

Jump to bottom
Andrey Talman edited this page Dec 28, 2022 · 25 revisions

Context

Validation framework is used to smoke test and validate PyTorch and Domain libraries on both CPU and GPU machines. Linux, Windows and MacOS (x86 and Apple Silicon) are supported. Following are the requirements for the validation framework:

  • Support Linux, Windows and MacOS using ephemeral runners with only minimal dependencies installed
  • Support CPU and GPU runners, with older Nvidia GPU driver to provide for backward compatibility tests (our CI runs on latest Nvidia Drivers)
  • Execute on nightly basis
  • Surface the result on HUD
  • Cover all the Released Domain Libraries
  • Follow same instructions as get started page for installation
  • Cover nightly, test and release channel
  • Use same matrix as PyTorch Core and Nova Project, so that after new build is introduced to PyTorch Core, it should become available for validation
  • Smoke test that will cover PyTorch standalone or PyTorch with all Domain Libraries

How we use Validation Framework

Validation framework is used in two different ways:

  • Nightly Validation of PyTorch, TorchAudio, TorchVision as one ecosystem. Using same instructions as in get started page. These workflows are implemented in validate-binaries.yml and are used for nightly and release validation.

  • Standalone Domain Library validation. Currently implemented for TorchText and TorchRec domain libaries and use

Binary build matrix

Binary build matrix contains current configuration that is supported by PyTorch core and domain libraries. Binaries build matrix is generated using the following workflow: generate_binary_build_matrix.yml. For additional details refer to documentation here

The oputput of the workflow above is JSON

Currently Following CUDA and Python configurations are supported:

CUDA CUDNN additional details
11.6 8.3.2.44 Stable CUDA Release
11.7 8.5.0.96 Latest CUDA Release
11.8 8.5.0.96 CUDA Release Supported on nightly
Python versions Package details
3.7 Supported on Conda and Pip
3.8 Supported on Conda and Pip
3.9 Supported on Conda and Pip
3.10 Supported on Conda and Pip
3.11 Supported on Pip only
Clone this wiki locally