Skip to content

Commit 6b92d87

Browse files
authored
Updation for inference docs page: migration to R2 (#2374)
* Updation for documentation page: migration to R2 * fix for llama2 assets download instructions
1 parent 55e41e0 commit 6b92d87

File tree

11 files changed

+50
-37
lines changed

11 files changed

+50
-37
lines changed

docs/benchmarks/automotive/3d_object_detection/get-pointpainting-data.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,16 +13,16 @@ The benchmark implementation run command will automatically download the preproc
1313

1414
=== "Validation"
1515

16-
### Get Validation Dataset
16+
### Get Validation and Calibration Dataset
1717
```
18-
mlcr get,dataset,waymo -j
18+
mlcr get,dataset,waymo,_r2-downloader,_mlc -j
1919
```
2020

2121
=== "Calibration"
2222

23-
### Get Calibration Dataset
23+
### Get Calibration Dataset only
2424
```
25-
mlcr get,dataset,waymo,calibration -j
25+
mlcr get,dataset,waymo,calibration,_r2-downloader,_mlc -j
2626
```
2727

2828
- `--outdirname=<PATH_TO_DOWNLOAD_WAYMO_DATASET>` could be provided to download the dataset to a specific location.
@@ -33,7 +33,7 @@ The benchmark implementation run command will automatically download the preproc
3333
The benchmark implementation run command will automatically download the model. In case you want to download only the PointPainting model, you can use the below command.
3434

3535
```bash
36-
mlcr get,ml-model,pointpainting -j
36+
mlcr get,ml-model,pointpainting,_r2-downloader,_mlc -j
3737
```
3838

3939
- `--outdirname=<PATH_TO_DOWNLOAD_POINTPAINTING_MODEL>` could be provided to download the model files to a specific location.

docs/benchmarks/graph/get-rgat-data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ Get the Official MLPerf R-GAT Model
4646

4747
### PyTorch
4848
```
49-
mlcr get,ml-model,rgat -j
49+
mlcr get,ml-model,rgat,_r2-downloader,_mlcommons -j
5050
```
5151

5252
- `--outdirname=<PATH_TO_DOWNLOAD_RGAT_MODEL>` could be provided to download the model to a specific location.

docs/benchmarks/language/get-deepseek-r1-data.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,16 @@ The benchmark implementation run command will automatically download the validat
2121
### Get Calibration Dataset
2222
```
2323
mlcr get,preprocessed,dataset,deepseek-r1,_calibration,_mlc,_rclone --outdirname=<path to download> -j
24-
```
24+
```
25+
26+
## Model
27+
The benchmark implementation run command will automatically download the required model and do the necessary conversions. In case you want to only download the official model, you can use the below commands.
28+
29+
=== "Pytorch"
30+
31+
=== "From MLCOMMONS Storage"
32+
33+
### Get the Official MLPerf DeekSeek-R1 model from MLCOMMONS Storage
34+
```
35+
mlcr get,ml-model,deepseek-r1,_r2-downloader,_mlc,_dry-run -j
36+
```

docs/benchmarks/language/get-gptj-data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ Get the Official MLPerf GPT-J Model
3636

3737
### Pytorch
3838
```
39-
mlcr get,ml-model,gptj,_pytorch -j
39+
mlcr get,ml-model,gptj,_fp32,_pytorch,_r2-downloader -j
4040
```
4141

4242
- `--outdirname=<PATH_TO_DOWNLOAD_GPTJ_MODEL>` could be provided to download the model to a specific location.

docs/benchmarks/language/get-llama2-70b-data.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -33,14 +33,15 @@ The benchmark implementation run command will automatically download the validat
3333

3434
### Get Preprocessed Validation Dataset
3535
```
36-
mlcr get,dataset,preprocessed,openorca,_validation -j
36+
mlcr get,dataset,preprocessed,openorca,_validation,_r2-downloader,_mlc -j
3737
```
3838

3939
=== "Calibration"
4040

4141
### Get Preprocessed Calibration dataset
4242
```
43-
mlcr get,dataset,preprocessed,openorca,_calibration -j
43+
mlcr get,dataset,preprocessed,openorca,_calibration,_r2-downloader,_mlc -j
44+
```
4445
```
4546

4647
=== "Unprocessed Dataset"
@@ -69,13 +70,13 @@ The benchmark implementation run command will automatically download the require
6970

7071
=== "Pytorch"
7172

72-
=== "From MLCOMMONS storage"
73-
74-
> **Note:** One has to accept the [MLCommons Llama 2 License Confidentiality Notice](https://llama2.mlcommons.org/) to access the model files in MLCOMMONS storage.
75-
76-
### Get the Official MLPerf LLAMA2-70B model from MLCOMMONS storage
73+
=== "From MLCOMMONS Storage"
74+
75+
> **Note:** One has to accept the [MLCommons Llama 2 License Confidentiality Notice](https://llama2.mlcommons.org/) to access the model files in MLCOMMONS Storage.
76+
77+
### Get the Official MLPerf LLAMA2-70B model from MLCOMMONS Storage
7778
```
78-
mlcr get,ml-model,llama2-70b,_fp32,_pytorch -j
79+
mlcr get,ml-model,llama2-70b,_pytorch,_r2-downloader,_70b,_mlc -j
7980
```
8081

8182
=== "From Hugging Face repo"

docs/benchmarks/language/get-llama3_1-405b-data.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@ The benchmark implementation run command will automatically download the validat
1313

1414
### Get Validation Dataset
1515
```
16-
mlcr get,dataset,mlperf,inference,llama3,_validation --outdirname=<path to download> -j
16+
mlcr get,dataset,mlperf,inference,llama3,_validation,_r2-downloader --outdirname=<path to download> -j
1717
```
1818

1919
=== "Calibration"
2020

2121
### Get Calibration Dataset
2222
```
23-
mlcr get,dataset,mlperf,inference,llama3,_calibration --outdirname=<path to download> -j
23+
mlcr get,dataset,mlperf,inference,llama3,_calibration,_r2-downloader --outdirname=<path to download> -j
2424
```
2525

2626
- `--outdirname=<PATH_TO_DOWNLOAD_LLAMA3_405B_DATASET>` could be provided to download the dataset to a specific location.
@@ -30,13 +30,13 @@ The benchmark implementation run command will automatically download the require
3030

3131
=== "Pytorch"
3232

33-
=== "From MLCOMMONS Google Drive"
33+
=== "From MLCOMMONS Storage"
3434

35-
> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Google Drive.
35+
> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Storage.
3636

3737
### Get the Official MLPerf LLAMA3.1-405B model from MLCOMMONS Google Drive
3838
```
39-
mlcr get,ml-model,llama3 -j
39+
mlcr get,ml-model,llama3,_mlc,_r2-downloader,_405b --outdirname=<path to download> -j
4040
```
4141

4242
=== "From Hugging Face repo"

docs/benchmarks/language/get-llama3_1-8b-data.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,21 +15,21 @@ The benchmark implementation run command will automatically download the validat
1515

1616
### Get Validation Dataset
1717
```
18-
mlcr get,dataset,cnndm,_validation,_datacenter,_llama3,_mlc,_rclone --outdirname=<path to download> -j
18+
mlcr get,dataset,cnndm,_validation,_datacenter,_llama3,_mlc,_r2-downloader --outdirname=<path to download> -j
1919
```
2020

2121
=== "5000 samples (Edge)"
2222

2323
### Get Validation Dataset
2424
```
25-
mlcr get,dataset,cnndm,_validation,_edge,_llama3,_mlc,_rclone --outdirname=<path to download> -j
25+
mlcr get,dataset,cnndm,_validation,_edge,_llama3,_mlc,_r2-downloader --outdirname=<path to download> -j
2626
```
2727

2828
=== "Calibration"
2929

3030
### Get Calibration Dataset
3131
```
32-
mlcr get,dataset,cnndm,_calibration,_llama3,_mlc,_rclone --outdirname=<path to download> -j
32+
mlcr get,dataset,cnndm,_calibration,_llama3,_mlc,_r2-downloader --outdirname=<path to download> -j
3333
```
3434

3535
- `--outdirname=<PATH_TO_DOWNLOAD_LLAMA3_405B_DATASET>` could be provided to download the dataset to a specific location.
@@ -39,13 +39,13 @@ The benchmark implementation run command will automatically download the require
3939

4040
=== "Pytorch"
4141

42-
=== "From MLCOMMONS Google Drive"
42+
=== "From MLCOMMONS Storage"
4343

44-
> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Google Drive.
44+
> **Note:** One has to accept the [MLCommons Llama 3.1 License Confidentiality Notice](http://llama3-1.mlcommons.org/) to access the model files in MLCOMMONS Storage.
4545

46-
### Get the Official MLPerf LLAMA3.1-405B model from MLCOMMONS Cloudfare R2
46+
### Get the Official MLPerf LLAMA3.1-8B model from MLCOMMONS Cloudfare R2
4747
```
48-
TBD
48+
mlcr get,ml-model,llama3,_mlc,_8b,_r2-downloader --outdirname=<path to download> -j
4949
```
5050

5151
=== "From Hugging Face repo"

docs/benchmarks/language/get-mixtral-8x7b-data.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,14 +12,14 @@ The benchmark implementation run command will automatically download the preproc
1212

1313
### Get Validation Dataset
1414
```
15-
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined -j
15+
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined,_r2-downloader,_validation -j
1616
```
1717

1818
=== "Calibration"
1919

2020
### Get Calibration Dataset
2121
```
22-
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined,_calibration -j
22+
mlcr get,dataset-mixtral,openorca-mbxp-gsm8k-combined,_r2-downloader,_calibration -j
2323
```
2424

2525
- `--outdirname=<PATH_TO_DOWNLOAD_MIXTRAL_DATASET>` could be provided to download the dataset to a specific location.
@@ -33,7 +33,7 @@ Get the Official MLPerf MIXTRAL-8x7b Model
3333

3434
### Pytorch
3535
```
36-
mlcr get,ml-model,mixtral -j
36+
mlcr get,ml-model,mixtral,_r2-downloader,_mlc -j
3737
```
3838

3939
- `--outdirname=<PATH_TO_DOWNLOAD_MIXTRAL_MODEL>` could be provided to download the model to a specific location.

docs/benchmarks/recommendation/get-dlrm-v2-data.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ The benchmark implementation run command will automatically download the validat
1414

1515
### Get Validation Dataset
1616
```
17-
mlcr get,dataset,criteo,_validation -j
17+
mlcr get,preprocessed,dataset,criteo,_r2-downloader,_mlc -j
1818
```
1919

2020
- `--outdirname=<PATH_TO_DOWNLOAD_CRITEO_DATASET>` could be provided to download the dataset to a specific location.
@@ -28,7 +28,7 @@ Get the Official MLPerf DLRM v2 Model
2828

2929
### Pytorch
3030
```
31-
mlcr get,ml-model,dlrm,_pytorch,_weight_sharded,_rclone -j
31+
mlcr get,ml-model,dlrm,_pytorch,_fp32,_weight_sharded,_r2-downloader -j
3232
```
3333

3434

docs/benchmarks/speech_to_text/get-whisper-data.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ The benchmark implementation run command will automatically download the validat
1515

1616
### Get Preprocessed Validation Dataset
1717
```
18-
mlcr get,dataset,whisper,_preprocessed,_mlc,_rclone --outdirname=<path to download> -j
18+
mlcr get,dataset,whisper,_preprocessed,_mlc,_r2-downloader --outdirname=<path to download> -j
1919
```
2020

2121
=== "Unprocessed"
@@ -34,7 +34,7 @@ The benchmark implementation run command will automatically download the require
3434

3535
### Get the Official MLPerf Whisper model from MLCOMMONS Cloudflare R2
3636
```
37-
mlcr get,ml-model,whisper,_rclone,_mlc s-j
37+
mlcr get,ml-model,whisper,_r2-downloader,_mlc -j
3838
```
3939

4040
- `--outdirname=<PATH_TO_DOWNLOAD_WHISPER_MODEL>` could be provided to download the model to a specific location.

0 commit comments

Comments
 (0)