Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ The benchmark implementation run command will automatically download the preproc

=== "Validation"

### Get Validation Dataset
### Get Validation and Calibration Dataset
```
mlcr get,dataset,waymo -j
mlcr get,dataset,waymo,_r2-downloader,_mlc -j
```

=== "Calibration"

### Get Calibration Dataset
### Get Calibration Dataset only
```
mlcr get,dataset,waymo,calibration -j
mlcr get,dataset,waymo,calibration,_r2-downloader,_mlc -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_WAYMO_DATASET>` could be provided to download the dataset to a specific location.
Expand All @@ -33,7 +33,7 @@ The benchmark implementation run command will automatically download the preproc
The benchmark implementation run command will automatically download the model. In case you want to download only the PointPainting model, you can use the below command.

```bash
mlcr get,ml-model,pointpainting -j
mlcr get,ml-model,pointpainting,_r2-downloader,_mlc -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_POINTPAINTING_MODEL>` could be provided to download the model files to a specific location.
2 changes: 1 addition & 1 deletion docs/benchmarks/graph/get-rgat-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Get the Official MLPerf R-GAT Model

### PyTorch
```
mlcr get,ml-model,rgat -j
mlcr get,ml-model,rgat,_r2-downloader,_mlcommons -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_RGAT_MODEL>` could be provided to download the model to a specific location.
14 changes: 13 additions & 1 deletion docs/benchmarks/language/get-deepseek-r1-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,16 @@ The benchmark implementation run command will automatically download the validat
### Get Calibration Dataset
```
mlcr get,preprocessed,dataset,deepseek-r1,_calibration,_mlc,_rclone --outdirname=<path to download> -j
```
```

## Model
The benchmark implementation run command will automatically download the required model and do the necessary conversions. In case you want to only download the official model, you can use the below commands.

=== "Pytorch"

=== "From MLCOMMONS Storage"

### Get the Official MLPerf DeekSeek-R1 model from MLCOMMONS Storage
```
mlcr get,ml-model,deepseek-r1,_r2-downloader,_mlc,_dry-run -j
```
2 changes: 1 addition & 1 deletion docs/benchmarks/language/get-gptj-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Get the Official MLPerf GPT-J Model

### Pytorch
```
mlcr get,ml-model,gptj,_pytorch -j
mlcr get,ml-model,gptj,_fp32,_pytorch,_r2-downloader -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_GPTJ_MODEL>` could be provided to download the model to a specific location.
17 changes: 9 additions & 8 deletions docs/benchmarks/language/get-llama2-70b-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,15 @@ The benchmark implementation run command will automatically download the validat

### Get Preprocessed Validation Dataset
```
mlcr get,dataset,preprocessed,openorca,_validation -j
mlcr get,dataset,preprocessed,openorca,_validation,_r2-downloader,_mlc -j
```

=== "Calibration"

### Get Preprocessed Calibration dataset
```
mlcr get,dataset,preprocessed,openorca,_calibration -j
mlcr get,dataset,preprocessed,openorca,_calibration,_r2-downloader,_mlc -j
```
```

=== "Unprocessed Dataset"
Expand Down Expand Up @@ -69,13 +70,13 @@ The benchmark implementation run command will automatically download the require

=== "Pytorch"

=== "From MLCOMMONS storage"

> **Note:** One has to accept the [MLCommons Llama 2 License Confidentiality Notice](https://llama2.mlcommons.org/) to access the model files in MLCOMMONS storage.

### Get the Official MLPerf LLAMA2-70B model from MLCOMMONS storage
=== "From MLCOMMONS Storage"
> **Note:** One has to accept the [MLCommons Llama 2 License Confidentiality Notice](https://llama2.mlcommons.org/) to access the model files in MLCOMMONS Storage.
### Get the Official MLPerf LLAMA2-70B model from MLCOMMONS Storage
```
mlcr get,ml-model,llama2-70b,_fp32,_pytorch -j
mlcr get,ml-model,llama2-70b,_pytorch,_r2-downloader,_70b,_mlc -j
```

=== "From Hugging Face repo"
Expand Down
10 changes: 5 additions & 5 deletions docs/benchmarks/language/get-llama3_1-405b-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ The benchmark implementation run command will automatically download the validat

### Get Validation Dataset
```
mlcr get,dataset,mlperf,inference,llama3,_validation --outdirname=<path to download> -j
mlcr get,dataset,mlperf,inference,llama3,_validation,_r2-downloader --outdirname=<path to download> -j
```

=== "Calibration"

### Get Calibration Dataset
```
mlcr get,dataset,mlperf,inference,llama3,_calibration --outdirname=<path to download> -j
mlcr get,dataset,mlperf,inference,llama3,_calibration,_r2-downloader --outdirname=<path to download> -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_LLAMA3_405B_DATASET>` could be provided to download the dataset to a specific location.
Expand All @@ -30,13 +30,13 @@ The benchmark implementation run command will automatically download the require

=== "Pytorch"

=== "From MLCOMMONS Google Drive"
=== "From MLCOMMONS Storage"

> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Google Drive.
> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Storage.

### Get the Official MLPerf LLAMA3.1-405B model from MLCOMMONS Google Drive
```
mlcr get,ml-model,llama3 -j
mlcr get,ml-model,llama3,_mlc,_r2-downloader,_405b --outdirname=<path to download> -j
```

=== "From Hugging Face repo"
Expand Down
14 changes: 7 additions & 7 deletions docs/benchmarks/language/get-llama3_1-8b-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,21 +15,21 @@ The benchmark implementation run command will automatically download the validat

### Get Validation Dataset
```
mlcr get,dataset,cnndm,_validation,_datacenter,_llama3,_mlc,_rclone --outdirname=<path to download> -j
mlcr get,dataset,cnndm,_validation,_datacenter,_llama3,_mlc,_r2-downloader --outdirname=<path to download> -j
```

=== "5000 samples (Edge)"

### Get Validation Dataset
```
mlcr get,dataset,cnndm,_validation,_edge,_llama3,_mlc,_rclone --outdirname=<path to download> -j
mlcr get,dataset,cnndm,_validation,_edge,_llama3,_mlc,_r2-downloader --outdirname=<path to download> -j
```

=== "Calibration"

### Get Calibration Dataset
```
mlcr get,dataset,cnndm,_calibration,_llama3,_mlc,_rclone --outdirname=<path to download> -j
mlcr get,dataset,cnndm,_calibration,_llama3,_mlc,_r2-downloader --outdirname=<path to download> -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_LLAMA3_405B_DATASET>` could be provided to download the dataset to a specific location.
Expand All @@ -39,13 +39,13 @@ The benchmark implementation run command will automatically download the require

=== "Pytorch"

=== "From MLCOMMONS Google Drive"
=== "From MLCOMMONS Storage"

> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Google Drive.
> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Storage.

### Get the Official MLPerf LLAMA3.1-405B model from MLCOMMONS Cloudfare R2
### Get the Official MLPerf LLAMA3.1-8B model from MLCOMMONS Cloudfare R2
```
TBD
mlcr get,ml-model,llama3,_mlc,_8b,_r2-downloader --outdirname=<path to download> -j
```

=== "From Hugging Face repo"
Expand Down
6 changes: 3 additions & 3 deletions docs/benchmarks/language/get-mixtral-8x7b-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ The benchmark implementation run command will automatically download the preproc

### Get Validation Dataset
```
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined -j
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined,_r2-downloader,_validation -j
```

=== "Calibration"

### Get Calibration Dataset
```
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined,_calibration -j
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined,_r2-downloader,_calibration -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_MIXTRAL_DATASET>` could be provided to download the dataset to a specific location.
Expand All @@ -33,7 +33,7 @@ Get the Official MLPerf MIXTRAL-8x7b Model

### Pytorch
```
mlcr get,ml-model,mixtral -j
mlcr get,ml-model,mixtral,_r2-downloader,_mlc -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_MIXTRAL_MODEL>` could be provided to download the model to a specific location.
4 changes: 2 additions & 2 deletions docs/benchmarks/recommendation/get-dlrm-v2-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The benchmark implementation run command will automatically download the validat

### Get Validation Dataset
```
mlcr get,dataset,criteo,_validation -j
mlcr get,preprocessed,dataset,criteo,_r2-downloader,_mlc -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_CRITEO_DATASET>` could be provided to download the dataset to a specific location.
Expand All @@ -28,7 +28,7 @@ Get the Official MLPerf DLRM v2 Model

### Pytorch
```
mlcr get,ml-model,dlrm,_pytorch,_weight_sharded,_rclone -j
mlcr get,ml-model,dlrm,_pytorch,_fp32,_weight_sharded,_r2-downloader -j
```


Expand Down
4 changes: 2 additions & 2 deletions docs/benchmarks/speech_to_text/get-whisper-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The benchmark implementation run command will automatically download the validat

### Get Preprocessed Validation Dataset
```
mlcr get,dataset,whisper,_preprocessed,_mlc,_rclone --outdirname=<path to download> -j
mlcr get,dataset,whisper,_preprocessed,_mlc,_r2-downloader --outdirname=<path to download> -j
```

=== "Unprocessed"
Expand All @@ -34,7 +34,7 @@ The benchmark implementation run command will automatically download the require

### Get the Official MLPerf Whisper model from MLCOMMONS Cloudflare R2
```
mlcr get,ml-model,whisper,_rclone,_mlc s-j
mlcr get,ml-model,whisper,_r2-downloader,_mlc -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_WHISPER_MODEL>` could be provided to download the model to a specific location.
4 changes: 2 additions & 2 deletions docs/benchmarks/text_to_image/get-sdxl-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,12 @@ Get the Official MLPerf Stable Diffusion Model
=== "FP 16"
### Pytorch
```
mlcr get,ml-model,sdxl,_pytorch,_fp16 -j
mlcr get,ml-model,sdxl,_pytorch,_fp16,_r2-downloader -j
```
=== "FP 32"
### Pytorch
```
mlcr get,ml-model,sdxl,_pytorch,_fp32 -j
mlcr get,ml-model,sdxl,_pytorch,_fp32,_r2-downloader -j
```

- `--outdirname=<PATH_TO_DOWNLOAD_SDXL_MODEL>` could be provided to download the model to a specific location.