Skip to content

Commit

Permalink
cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
Titus-von-Koeller authored Sep 21, 2024
1 parent f919155 commit a785c72
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/source/non_cuda_backends.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ Thank you for your support!

The following performance data is collected from Intel 4th Gen Xeon (SPR) platform. The tables show speed-up and memory compared with different data types of [Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf).

#### Inference Table:
#### Inference (CPU)

| Data Type | BF16 | INT8 | NF4 | FP4 |
|---|---|---|---|---|
| Speed-Up (vs BF16) | 1.0x | 0.6x | 2.3x | 0.03x |
| Memory (GB) | 13.1 | 7.6 | 5.0 | 4.6 |

#### Fine-Tuning Table:
#### Fine-Tuning (CPU)

| Data Type | AMP BF16 | INT8 | NF4 | FP4 |
|---|---|---|---|---|
Expand Down

0 comments on commit a785c72

Please sign in to comment.