Skip to content

[SGLANG] add sglang block fp8 gemm kernels into benchmark #3676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
vlad-penkin opened this issue Mar 14, 2025 · 4 comments · May be fixed by #3645 or #3796
Open

[SGLANG] add sglang block fp8 gemm kernels into benchmark #3676

vlad-penkin opened this issue Mar 14, 2025 · 4 comments · May be fixed by #3645 or #3796
Assignees
@LiyangLingIntel
Labels
codegen: gemm performance reporting tests: ecosystem
Milestone
5. Ecosystem enab...

Comments

@vlad-penkin
Copy link
Contributor

No description provided.

@vlad-penkin vlad-penkin linked a pull request Mar 14, 2025 that will close this issue
[SGLANG] add sglang block fp8 gemm kernels into benchmark #3645
Open
7 tasks
@vlad-penkin
Copy link
Contributor Author

@airMeng i've added this ticket to track this feature. Our repo policy is to have linked issue for each open PR.

@vlad-penkin
Copy link
Contributor Author

@LiyangLingIntel can we close this ticket?

@LiyangLingIntel
Copy link
Contributor

@LiyangLingIntel can we close this ticket?

I suggest to merge #3654 to third-party benchmark when #3796 lands.
SGLang fp8_kernel depends on sgl-kernel, while sgl-kernel is hard-coupled with NV backend.

@LiyangLingIntel
Copy link
Contributor

@LiyangLingIntel can we close this ticket?

I suggest to merge #3654 to third-party benchmark when #3796 lands. SGLang fp8_kernel depends on sgl-kernel, while sgl-kernel is hard-coupled with NV backend.

Cherry pick block_fp8_gemm kernel from #3645 to #3796, block fp8 gemm benchmark will be integrated in sglang third party benchmark with #3796.

@LiyangLingIntel LiyangLingIntel linked a pull request May 15, 2025 that will close this issue
[benchmarks][ci] Initial integration of sglang kernels to benchmarks #3796
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@LiyangLingIntel LiyangLingIntel

Labels
codegen: gemm performance reporting tests: ecosystem
Projects
None yet
Milestone
5. Ecosystem enabling and performance
2 participants
@LiyangLingIntel @vlad-penkin