Ultimate table of Quant Types

[Aver00]

Last updated 19.02.2026

All GGUF quant types

code▲	name	bpw	Method	imatrix	llama.cpp	ik⁠-⁠llama	Comment
0	f32	32.0	Casting	n/a	✅	✅
1	f16	16.0	Casting	n/a	✅	✅	can repack to bf16_r16
2	q4_0	4.5	Legacy, symmetric	Optional	✅	✅	can repack to q4_0_r8
3	q4_1	5.0	Legacy, asymmetric	Optional	✅	✅
6	q5_0	5.5	Legacy, symmetric	Optional	✅	✅	can repack to q5_0_r4
7	q5_1	6.0	Legacy, asymmetric	Optional	✅	✅
8	q8_0	8.5	Legacy, symmetric	Not used	✅	✅	can repack to q8_0_r8
9	q8_1	9.0	Legacy, asymmetric	n/a	❌	❌	internal type
10	q2_K	2.625	K-quant, asymmetric	Required	✅	✅	can repack to q2_k_r4
11	q3_K	3.4375	K-quant, symmetric	Optional	✅	✅	can repack to q3_k_r4
12	q4_K	4.5	K-quant, asymmetric	Optional	✅	✅	can repack to q4_k_r4
13	q5_K	5.5	K-quant, asymmetric	Optional	✅	✅	can repack to q5_k_r4
14	q6_K	6.5625	K-quant, symmetric	Optional	✅	✅	can repack to q6_k_r4
15	q8_K	-		n/a	❌	❌	internal type
16	iq2_xxs	2.0625	I-quant	Required	✅	✅	can repack to iq2_xxs_r4
17	iq2_xs	2.3125	I-quant	Required	✅	✅	can repack to iq2_xs_r4
18	iq3_xxs	3.0625	I-quant	Optional	✅	✅	can repack to iq3_xxs_r4
19	iq1_s	1.5625	I-quant	Required	✅	✅
20	iq4_nl	4.5	Non-linear	Optional	✅	✅	can repack to iq4_nl_r4
21	iq3_s	3.4375	I-quant	Optional	✅	✅	can repack to iq3_s_r4
22	iq2_s	2.5625	I-quant	Required	✅	✅	can repack to iq2_s_r4
23	iq4_xs	4.25	Non-linear	Optional	✅	✅	can repack to iq4_xs_r8
24	i8	8.0	Integer	n/a	❌	❌
25	i16	16.0	Integer	n/a	❌	❌
26	i32	32.0	Integer	n/a	❌	❌
27	i64	64.0	Integer	n/a	❌	❌
28	f64	64.0	Casting	n/a	❌	❌
29	iq1_m	1.75	I-quant	Required	✅	✅
30	bf16	16.0	Casting	n/a	✅	✅	can repack to bf16_r16
34	tq1_0	1.6875	Ternary	Not used	✅	❌
35	tq2_0	2.0625	Ternary	Not used	✅	❌
36	i2_s	2.0	Ternary	n/a	❌	❌	read into iq2_bn
39	mxfp4	4.25	Microscaling, symmetric	Not used	✅	✅
97	q8_0_x4	8.5		Not used	❌	❌	internal type
98	q8_1_x4	9.0		Not used	❌	❌	internal type
99	q8_2_x4	9.0		Not used	❌	❌	internal type
133	q6_0	6.5	Legacy, symmetric	Optional	❌	✅	can repack to q6_0_r4
134	iq1_bn	1.625 + 16⁠/⁠row_size	Ternary	Not used	❌	✅
135	iq2_bn	2.0 + 32⁠/⁠row_size	Ternary	Not used	❌	✅	can repack to iq2_bn_r4
136	q8_K64	-		n/a	❌	❌	internal type
137	iq2_k	2.375	I-quant	Optional	❌	✅	can repack to iq2_k_r4
138	iq3_k	3.4375	I-quant	Optional	❌	✅	can repack to iq3_k_r4
139	iq4_k	4.5	I-quant	Optional	❌	✅	can repack to iq4_k_r4
140	iq5_k	5.5	I-quant	Optional	❌	✅	can repack to iq5_k_r4
141	iq6_k	6.625	I-quant	Optional	❌	✅
144	iq4_ks	4.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	can repack to iq4_ks_r4
145	iq2_ks	2.1875 + 16⁠/⁠row_size	I-quant	Optional	❌	✅
146	iq4_kss	4.0 + 32⁠/⁠row_size	I-quant	Optional	❌	✅
147	q8_K16	-		n/a	❌	❌	internal type
148	q8_K32	-		n/a	❌	❌	internal type
149	q8_KR8	-		n/a	❌	❌	internal type
150	q8_K128	-		n/a	❌	❌	internal type
151	q8_KV	8.0 + 64⁠/⁠row_size		Not used	❌	✅
152	iq5_ks	5.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	can repack to iq5_ks_r4
153	iq2_kt	2.125 + 32⁠/⁠row_size	Trellis	Required	❌	✅
154	iq3_kt	3.125 + 32⁠/⁠row_size	Trellis	Optional	❌	✅
155	iq4_kt	4.0 + 32⁠/⁠row_size	Trellis	Optional	❌	✅
156	iq3_ks	3.1875 + 16⁠/⁠row_size	I-quant	Optional	❌	✅
157	iq2_kl	2.6875 + 16⁠/⁠row_size	I-quant	Optional	❌	✅
158	iq1_kt	1.75 + 32⁠/⁠row_size	Trellis	Required	❌	✅
202	q4_0_r8	4.5	Legacy, symmetric	Optional	❌	✅	row-interleaved q4_0
206	q5_0_r4	5.5	Legacy, symmetric	Optional	❌	✅	row-interleaved q5_0
208	q8_0_r8	8.5	Legacy, symmetric	Not used	❌	✅	row-interleaved q8_0
210	q2_k_r4	2.625	K-quant, asymmetric	Optional	❌	✅	row-interleaved q2_K
211	q3_k_r4	3.4375	K-quant, symmetric	Optional	❌	✅	row-interleaved q3_K
212	q4_k_r4	4.5	K-quant, asymmetric	Optional	❌	✅	row-interleaved q4_K
213	q5_k_r4	5.5	K-quant, asymmetric	Optional	❌	✅	row-interleaved q5_K
214	q6_k_r4	6.5625	K-quant, symmetric	Optional	❌	✅	row-interleaved q6_K
216	iq2_xxs_r4	2.0625	I-quant	Required	❌	✅	row-interleaved iq2_xxs
217	iq2_xs_r4	2.3125	I-quant	Required	❌	✅	row-interleaved iq2_xs
218	iq3_xxs_r4	3.0625	I-quant	Optional	❌	✅	row-interleaved iq3_xxs
219	iq1_s_r4	1.5 + 16⁠/⁠row_size	I-quant	Required	❌	✅	lower-bpw row-interleaved iq1_s
220	iq4_nl_r4	4.5	Non-linear	Optional	❌	✅	row-interleaved iq4_nl
221	iq3_s_r4	3.4375	I-quant	Optional	❌	✅	row-interleaved iq3_s
222	iq2_s_r4	2.5625	I-quant	Required	❌	✅	row-interleaved iq2_s
223	iq4_xs_r8	4.25	Non-linear	Optional	❌	✅	row-interleaved iq4_xs
229	iq1_m_r4	1.75 + 16⁠/⁠row_size	I-quant	Required	❌	✅	higher-bpw row-interleaved iq1_m
230	bf16_r16	16.0	Casting	n/a	❌	✅	row-interleaved bf16
233	q6_0_r4	6.5	Legacy, symmetric	Optional	❌	✅	row-interleaved q6_0
335	iq2_bn_r4	2.0 + 32⁠/⁠row_size	Ternary	Not used	❌	✅	row-interleaved iq2_bn
337	iq2_k_r4	2.375	I-quant	Optional	❌	✅	row-interleaved iq2_k
338	iq3_k_r4	3.4375	I-quant	Optional	❌	✅	row-interleaved iq3_k
339	iq4_k_r4	4.5	I-quant	Optional	❌	✅	row-interleaved iq4_k
340	iq5_k_r4	5.5	I-quant	Optional	❌	✅	row-interleaved iq5_k
344	iq4_ks_r4	4.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	row-interleaved iq4_ks
352	iq5_ks_r4	5.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	row-interleaved iq5_ks
397	q8_k_r16	8.0625		Not used	❌	❌	row-interleaved q8_K
398	q8_KV_r8	8.0 + 32⁠/⁠row_size		Not used	❌	✅	lower-bpw row-interleaved q8_KV
399	q8_k_r8	8.0625		Not used	❌	✅	row-interleaved q8_K

The table combines information from llama.cpp and ik-llama.cpp source code.

bpw (bits-per-weight) of certain quant types depends on the first dimension of quantized tensor (row_size).

Same table sorted by bpw

code	name	bpw▲	Method	imatrix	llama.cpp	ik⁠-⁠llama	Comment
219	iq1_s_r4	1.5 + 16⁠/⁠row_size	I-quant	Required	❌	✅	lower-bpw row-interleaved iq1_s
19	iq1_s	1.5625	I-quant	Required	✅	✅
134	iq1_bn	1.625 + 16⁠/⁠row_size	Ternary	Not used	❌	✅
34	tq1_0	1.6875	Ternary	Not used	✅	❌
29	iq1_m	1.75	I-quant	Required	✅	✅
229	iq1_m_r4	1.75 + 16⁠/⁠row_size	I-quant	Required	❌	✅	higher-bpw row-interleaved iq1_m
158	iq1_kt	1.75 + 32⁠/⁠row_size	Trellis	Required	❌	✅
36	i2_s	2.0	Ternary	n/a	❌	❌	read into iq2_bn
135	iq2_bn	2.0 + 32⁠/⁠row_size	Ternary	Not used	❌	✅	can repack to iq2_bn_r4
335	iq2_bn_r4	2.0 + 32⁠/⁠row_size	Ternary	Not used	❌	✅	row-interleaved iq2_bn
16	iq2_xxs	2.0625	I-quant	Required	✅	✅	can repack to iq2_xxs_r4
35	tq2_0	2.0625	Ternary	Not used	✅	❌
216	iq2_xxs_r4	2.0625	I-quant	Required	❌	✅	row-interleaved iq2_xxs
153	iq2_kt	2.125 + 32⁠/⁠row_size	Trellis	Required	❌	✅
145	iq2_ks	2.1875 + 16⁠/⁠row_size	I-quant	Optional	❌	✅
17	iq2_xs	2.3125	I-quant	Required	✅	✅	can repack to iq2_xs_r4
217	iq2_xs_r4	2.3125	I-quant	Required	❌	✅	row-interleaved iq2_xs
137	iq2_k	2.375	I-quant	Optional	❌	✅	can repack to iq2_k_r4
337	iq2_k_r4	2.375	I-quant	Optional	❌	✅	row-interleaved iq2_k
22	iq2_s	2.5625	I-quant	Required	✅	✅	can repack to iq2_s_r4
222	iq2_s_r4	2.5625	I-quant	Required	❌	✅	row-interleaved iq2_s
10	q2_K	2.625	K-quant, asymmetric	Required	✅	✅	can repack to q2_k_r4
210	q2_k_r4	2.625	K-quant, asymmetric	Optional	❌	✅	row-interleaved q2_K
157	iq2_kl	2.6875 + 16⁠/⁠row_size	I-quant	Optional	❌	✅
18	iq3_xxs	3.0625	I-quant	Optional	✅	✅	can repack to iq3_xxs_r4
218	iq3_xxs_r4	3.0625	I-quant	Optional	❌	✅	row-interleaved iq3_xxs
154	iq3_kt	3.125 + 32⁠/⁠row_size	Trellis	Optional	❌	✅
156	iq3_ks	3.1875 + 16⁠/⁠row_size	I-quant	Optional	❌	✅
11	q3_K	3.4375	K-quant, symmetric	Optional	✅	✅	can repack to q3_k_r4
21	iq3_s	3.4375	I-quant	Optional	✅	✅	can repack to iq3_s_r4
138	iq3_k	3.4375	I-quant	Optional	❌	✅	can repack to iq3_k_r4
211	q3_k_r4	3.4375	K-quant, symmetric	Optional	❌	✅	row-interleaved q3_K
221	iq3_s_r4	3.4375	I-quant	Optional	❌	✅	row-interleaved iq3_s
338	iq3_k_r4	3.4375	I-quant	Optional	❌	✅	row-interleaved iq3_k
146	iq4_kss	4.0 + 32⁠/⁠row_size	I-quant	Optional	❌	✅
155	iq4_kt	4.0 + 32⁠/⁠row_size	Trellis	Optional	❌	✅
23	iq4_xs	4.25	Non-linear	Optional	✅	✅	can repack to iq4_xs_r8
39	mxfp4	4.25	Microscaling, symmetric	Not used	✅	✅
223	iq4_xs_r8	4.25	Non-linear	Optional	❌	✅	row-interleaved iq4_xs
144	iq4_ks	4.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	can repack to iq4_ks_r4
344	iq4_ks_r4	4.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	row-interleaved iq4_ks
2	q4_0	4.5	Legacy, symmetric	Optional	✅	✅	can repack to q4_0_r8
12	q4_K	4.5	K-quant, asymmetric	Optional	✅	✅	can repack to q4_k_r4
20	iq4_nl	4.5	Non-linear	Optional	✅	✅	can repack to iq4_nl_r4
139	iq4_k	4.5	I-quant	Optional	❌	✅	can repack to iq4_k_r4
202	q4_0_r8	4.5	Legacy, symmetric	Optional	❌	✅	row-interleaved q4_0
212	q4_k_r4	4.5	K-quant, asymmetric	Optional	❌	✅	row-interleaved q4_K
220	iq4_nl_r4	4.5	Non-linear	Optional	❌	✅	row-interleaved iq4_nl
339	iq4_k_r4	4.5	I-quant	Optional	❌	✅	row-interleaved iq4_k
3	q4_1	5.0	Legacy, asymmetric	Optional	✅	✅
152	iq5_ks	5.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	can repack to iq5_ks_r4
352	iq5_ks_r4	5.25 + 32⁠/⁠row_size	I-quant	Optional	❌	✅	row-interleaved iq5_ks
6	q5_0	5.5	Legacy, symmetric	Optional	✅	✅	can repack to q5_0_r4
13	q5_K	5.5	K-quant, asymmetric	Optional	✅	✅	can repack to q5_k_r4
140	iq5_k	5.5	I-quant	Optional	❌	✅	can repack to iq5_k_r4
206	q5_0_r4	5.5	Legacy, symmetric	Optional	❌	✅	row-interleaved q5_0
213	q5_k_r4	5.5	K-quant, asymmetric	Optional	❌	✅	row-interleaved q5_K
340	iq5_k_r4	5.5	I-quant	Optional	❌	✅	row-interleaved iq5_k
7	q5_1	6.0	Legacy, asymmetric	Optional	✅	✅
133	q6_0	6.5	Legacy, symmetric	Optional	❌	✅	can repack to q6_0_r4
233	q6_0_r4	6.5	Legacy, symmetric	Optional	❌	✅	row-interleaved q6_0
14	q6_K	6.5625	K-quant, symmetric	Optional	✅	✅	can repack to q6_k_r4
214	q6_k_r4	6.5625	K-quant, symmetric	Optional	❌	✅	row-interleaved q6_K
141	iq6_k	6.625	I-quant	Optional	❌	✅
24	i8	8.0	Integer	n/a	❌	❌
398	q8_KV_r8	8.0 + 32⁠/⁠row_size		Not used	❌	✅	lower-bpw row-interleaved q8_KV
151	q8_KV	8.0 + 64⁠/⁠row_size		Not used	❌	✅
397	q8_k_r16	8.0625		Not used	❌	❌	row-interleaved q8_K
399	q8_k_r8	8.0625		Not used	❌	✅	row-interleaved q8_K
8	q8_0	8.5	Legacy, symmetric	Not used	✅	✅	can repack to q8_0_r8
97	q8_0_x4	8.5		Not used	❌	❌	internal type
208	q8_0_r8	8.5	Legacy, symmetric	Not used	❌	✅	row-interleaved q8_0
9	q8_1	9.0	Legacy, asymmetric	n/a	❌	❌	internal type
98	q8_1_x4	9.0		Not used	❌	❌	internal type
99	q8_2_x4	9.0		Not used	❌	❌	internal type
1	f16	16.0	Casting	n/a	✅	✅	can repack to bf16_r16
25	i16	16.0	Integer	n/a	❌	❌
30	bf16	16.0	Casting	n/a	✅	✅	can repack to bf16_r16
230	bf16_r16	16.0	Casting	n/a	❌	✅	row-interleaved bf16
0	f32	32.0	Casting	n/a	✅	✅
26	i32	32.0	Integer	n/a	❌	❌
27	i64	64.0	Integer	n/a	❌	❌
28	f64	64.0	Casting	n/a	❌	❌
15	q8_K	-		n/a	❌	❌	internal type
136	q8_K64	-		n/a	❌	❌	internal type
147	q8_K16	-		n/a	❌	❌	internal type
148	q8_K32	-		n/a	❌	❌	internal type
149	q8_KR8	-		n/a	❌	❌	internal type
150	q8_K128	-		n/a	❌	❌	internal type

[Aver00]

There was a lot of vagueness about GGUF quant types. People often confuse model file types with tensor quntization types.

• Tensor quntization type (quant type) is a basic building block. It's the actual thing that matters for us, makers of custom quants.
• Model file type is just a recipe hardcoded in llama-quantize.cpp for lazy cooking. For example, IQ3_M GGUFs apply IQ3_S quant type to most of tensors, except for ffn_down they use IQ4_K quant type, and IQ5_K quant type for output/attn_v/attn_output tensors. There's no IQ3_M quant type.
  Since we use llama-quantize with --custom-q switch to specify custom rules, we don't care about IQ3_M.

If you find a mistake in the table or have something to add, please comment here.

@Thireus, @ubergarm, @ikawrakow

[ikawrakow]

The row scale is not always f16. Look for row_meta_size in ggml.c.

The minimum tensor row size I have seen is 512. But I never look at tiny models used for embeddings, so maybe there could be smaller row sizes there. The maximum row_size I have seen is 30k or some such. Definitely 64k is a safe upper bound.

Wait, is it the actual reason you don't want to port your quant types to mainline?

Mainline cannot handle tensor meta data, but that could be ported, so it is not the reason. I just don't contribute to llama.cpp any longer. If you take a look at the top-10 llama.cpp contributors, you will notice that 4 out of them no longer contribute to llama.cpp. I'm still in the top-10, despite not having contributed since March of 2024. This should give you some food for thought.

[Thireus]

@ikawrakow - For a given quant type, I need to predict the exact size of the quantized tensors (and if quantization is possible) without quantizing it. Is there an simple way for me to print this information from llama-quantize while skipping the actual quantization process?

In short, running llama-quantize against the BF16 model but only printing information about potential fallback and resulting quant size - a form of dry-run.

Edit: I believe I've figured out I can simply comment the quantize_row_* sections as such, but not sure about fallback yet:

[ikawrakow]

Short of changing the code no, there is no way to do a dry run with llama-quantize. Being able to do a dry run cleanly seems like an useful option to have. You can add a feature request if you like.

[Thireus]

Thank you @ikawrakow. I dug into the code and I'm a bit confused about these lines: https://github.com/ikawrakow/ik_llama.cpp/blob/main/src/llama-quantize.cpp#L1437-L1438

            if (!params->ignore_imatrix_rules && !imatrix &&
               (new_type == GGML_TYPE_IQ2_XXS ||
                new_type == GGML_TYPE_IQ2_XXS_R4 ||
                new_type == GGML_TYPE_IQ2_XS  ||
                new_type == GGML_TYPE_IQ2_XS_R4  ||
                new_type == GGML_TYPE_IQ2_S   ||
                new_type == GGML_TYPE_IQ2_S_R4||
                new_type == GGML_TYPE_IQ1_S   ||
                new_type == GGML_TYPE_IQ1_S_R4||
                new_type == GGML_TYPE_IQ1_M_R4||
               (new_type == GGML_TYPE_IQ1_M && strcmp(tensor->name, "token_embd.weight") && strcmp(tensor->name, "output.weight"))  ||
               (new_type == GGML_TYPE_Q2_K && ftype == LLAMA_FTYPE_MOSTLY_Q2_K_S && strcmp(tensor->name, "token_embd.weight") != 0))) {
               LLAMA_LOG_ERROR("\n\n============================================================\n");
               LLAMA_LOG_ERROR("Missing importance matrix for tensor %s in a very low-bit quantization\n", tensor->name);
               LLAMA_LOG_ERROR("The result will be garbage, so bailing out\n");
               LLAMA_LOG_ERROR("============================================================\n\n");
               throw std::runtime_error(format("Missing importance matrix for tensor %s in a very low-bit quantization", tensor->name));
           }

Specifically, the new_type == GGML_TYPE_IQ1_M && strcmp(tensor->name, "token_embd.weight") && strcmp(tensor->name, "output.weight") statement. And why GGML_TYPE_IQ1_M isn't in the same condition as GGML_TYPE_IQ1_M_R4 above new_type == GGML_TYPE_IQ1_M_R4||.

It used to be like this:

https://github.com/ikawrakow/ik_llama.cpp/blob/e23b2a7cc90c675ac8f1b32f7c86a1c8f39a03be/examples/quantize/quantize.cpp#L604

For ref. ikawrakow/ik_llama.cpp#823 (comment)

[Thireus]

Answered and fixed here: ikawrakow/ik_llama.cpp#1281

[Aver00]

Big update:

• calculate exact bpw for everything
• add "storage info" tooltips
• add "Method" column
• add "imatrix" column. "Required" means llama-quantize won't quantize to those types without an imatrix or explicit --ignore-imatrix-rules switch
• add "Comment" column for references to/from row-interleaved types
• remove "ggml_type enum" column
• add a copy of the table sorted by bpw

[Thireus]

Thank you @Aver00 - I'm currently working towards solely relying on the bf16 map file. Your table is going to be very helpful to verify the checks I've ported to the python scripts I've implemented. quant_assign.py also has additional options now to compute map files 73e1f2f - stay tuned - I still need to port things on the webui and enrich existing .map files with imatrix tensor attributes.

[Thireus]

I have now integrated and fixed the missing "+ N/⁠row_size" bpw for the relevant tensors. The tools of this tool suite are now calculating the correct size for the tensors of these special quantization types. See #67

Thank you @Aver00 - your table was very helpful to confirm and validate the issue.

[Aver00]

I remember tensors.map reported wrong sizes (#53 (reply in thread)) though somehow I didn't want to press on.