Skip to content

Commit 99657ed

Browse files
authored
Fix: Verify scales are not None for Cutlass FP8 FusedMoE (#1961)
## 📌 Description Verify quant scales for fp8 are non null in cutlass FusedMoE path. Currently, if these tensors are passed as None from python it will result in segmentation fault. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Enhanced validation for FP8 quantization parameters to improve system robustness and prevent potential null reference issues during quantization operations, reducing the risk of runtime errors when processing quantized model data. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Amir Klein <[email protected]>
1 parent d4a3ff4 commit 99657ed

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -822,6 +822,13 @@ class FusedMoeRunner : public tvm::ffi::ModuleObj {
822822
auto const fc2_dequant = quant_scales.value()[2];
823823
auto const fc1_input_dequant = quant_scales.value()[3];
824824

825+
TVM_FFI_ICHECK(fc1_dequant.get() != nullptr) << "Expecting fc1_dequant to be non null";
826+
TVM_FFI_ICHECK(fc2_quant.get() != nullptr) << "Expecting fc2_quant to be non null";
827+
TVM_FFI_ICHECK(fc2_dequant.get() != nullptr)
828+
<< "Expecting fc2_dequant_dequant to be non null";
829+
TVM_FFI_ICHECK(fc1_input_dequant.get() != nullptr)
830+
<< "Expecting fc1_input_dequant to be non null";
831+
825832
// Check types
826833
CHECK_INPUT_TYPE(fc1_dequant, dl_float32);
827834
CHECK_INPUT_TYPE(fc2_quant, dl_float32);

0 commit comments

Comments
 (0)