Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

upstream_merge_25_03_12 #478

Merged
merged 51 commits into from
Mar 12, 2025
Merged

upstream_merge_25_03_12 #478

merged 51 commits into from
Mar 12, 2025

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Mar 12, 2025

Out of schedule upstream merge to bring in the unified FP8 for MI and Navi

hmellor and others added 30 commits March 10, 2025 15:53
@hmellor
Correct capitalisation: Github -> GitHub (vllm-project#14561)
39be303 
Signed-off-by: Harry Mellor <[email protected]>
@ywang96
[V1][Bugfix] Fix handing of second_per_grid_ts for Qwen2-VL & Qwen2…
dea985a 
….5-VL (vllm-project#14548)

Signed-off-by: Roger Wang <[email protected]>
@hmellor
Correct capitalisation: VLLM -> vLLM (vllm-project#14562)
3b352a2 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
[Docs] Make installation URLs nicer (vllm-project#14556)
bc2d447 
Signed-off-by: Harry Mellor <[email protected]>
@chaunceyjiang @DarkLight1337
[Bugfix][v1] fixed llava-hf/llava-1.5-7b-hf is broken on V1 (vllm-pro…
92b0ce2 
…ject#14554)

Signed-off-by: chaunceyjiang <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@simon-mo
[Perf] Improve MLA on V1 (vllm-project#14540)
fb0acb6 
Signed-off-by: simon-mo <[email protected]>
@WoosukKwon
[Minor] Update the tqdm bar for parallel sampling (vllm-project#14571)
0967110 
Signed-off-by: Woosuk Kwon <[email protected]>
@varun-sundar-rabindranath
[V1] LoRA - Add triton kernels for V1 (vllm-project#13096)
5ff0d32 
Signed-off-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@russellb
Fix typo in benchmark_serving_structured_output.py (vllm-project#14566)
432d6da 
Signed-off-by: Russell Bryant <[email protected]>
@russellb
[V1] Prevent xgrammar from breaking TPU support (vllm-project#14575)
04421df 
Signed-off-by: Russell Bryant <[email protected]>
@jinzhen-lin @mgoin
[Kernel] moe wna16 cuda kernel (vllm-project#13321)
90e88ab 
Signed-off-by: Jinzhen Lin <[email protected]>
Co-authored-by: mgoin <[email protected]>
@comaniac
[MISC][V1] Handle exception of current_platform.get_device_name() in …
485afdd 
…arg_utils (vllm-project#14379)

Signed-off-by: Cody Yu <[email protected]>
@gnovack
[Neuron] Add Neuron device communicator for vLLM v1 (vllm-project#14085)
d612317
@liangfu
[neuron] add reshape_and_cache (vllm-project#14391)
c91b64f
@comaniac
[V1][PP] Do not block engine core when no requests to schedule (vllm-…
4290b70 
…project#14585)

Signed-off-by: Cody Yu <[email protected]>
@Concurrensee
[Bugfix] Fix FP16 overflow for DeepSeek V2 (vllm-project#13232)
c982ac5 
Signed-off-by: Yida Wu <[email protected]>
@ywang96 @varun-sundar-rabindranath
[V1][Core] Fix memory issue with logits & sampling (vllm-project#14508)
1fc973c 
Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@Isotr0py
[Misc] Correct deepseek-vl2 chat template (vllm-project#14558)
63d635d 
Signed-off-by: Isotr0py <[email protected]>
@cynthieye
[Perf]:Optimize qwen2-vl to reduce cudaMemcpyAsync (vllm-project#14377)
70b808f 
Signed-off-by: cynthieye <[email protected]>
@Isotr0py
[VLM] Cleanup siglip legacy code and fix broken paligemma multimodal …
1477ffc 
…processor (vllm-project#14602)

Signed-off-by: Isotr0py <[email protected]>
@russellb
benchmarks: simplify test jsonschema (vllm-project#14567)
08a1a11 
Signed-off-by: Russell Bryant <[email protected]>
@jeffdaily
dynamic distpatch of fp8 kernels (vllm-project#14245)
a1c8f37 
Signed-off-by: Jeff Daily <[email protected]>
@DarkLight1337
[Bugfix] Update --hf-overrides for Alibaba-NLP/gte-Qwen2 (vllm-pr…
af295e9 
…oject#14609)

Signed-off-by: DarkLight1337 <[email protected]>
@richardsliu
Uninstall dependencies before installing requirements/tpu.txt (vllm-p…
9ab3267 
…roject#14586)

Signed-off-by: <[email protected]>
Signed-off-by: Richard Liu <[email protected]>
@russellb
[V1] Add regex structured output support with xgrammar (vllm-project#…
4bf82d4 
…14590)

Signed-off-by: Russell Bryant <[email protected]>
@dilipgb @hmellor
docs: Add documentation for s390x cpu implementation (vllm-project#14198
07964e2 
)

Signed-off-by: Dilip Gowda Bhagavan <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
@LucasWilkinson
[BugFix/Build] Fix sparse kernels not getting built on hopper (vllm-p…
07b4b7a 
…roject#14572)

Signed-off-by: Lucas Wilkinson <[email protected]>
@jikunshang
[Hardware][Intel GPU] upgrade IPEX dependency to 2.6.10. (vllm-projec…
c6e14a6 
…t#14564)

Signed-off-by: Kunshang Ji <[email protected]>
@russellb
[V1] Remove cache from StructuredOutputManager (vllm-project#14622)
4cbf286 
Signed-off-by: Russell Bryant <[email protected]>
@hackty
fix some typos : supported_head_sizes (vllm-project#14627)
5305673
richardsliu and others added 20 commits March 11, 2025 21:14
@richardsliu
Fix run_tpu_test (vllm-project#14641)
d374f04 
Signed-off-by: <[email protected]>
Signed-off-by: Richard Liu <[email protected]>
@vanbasten23
[V1][TPU] Pad the block_table.shape[1] so the ragged paged attention …
863d315 
…can handle correctly (vllm-project#14597)
@comaniac
[Bugfix][V1][PP] Only warmup sampler at last PP rank (vllm-project#14643
b706d89 
)

Signed-off-by: Cody Yu <[email protected]>
@khluu
[release] Add commands to clean up logs on TPU release node (vllm-pro…
9f583e3 
…ject#14642)
@randyjhc @comaniac
[Feature] Add vllm bench CLI (vllm-project#13993)
36e0c8f 
Signed-off-by: Randy Chen <[email protected]>
Signed-off-by: Cody Yu <[email protected]>
Co-authored-by: Cody Yu <[email protected]>
@joerunde
[core][V1] pluggable scheduler (vllm-project#14466)
47532cd 
Signed-off-by: Joe Runde <[email protected]>
@JenZhao @ywang96
[Doc] Update benchmarks README (vllm-project#14646)
4a42b9f 
Signed-off-by: Jennifer Zhao <[email protected]>
Co-authored-by: Jennifer Zhao <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@farzadab
[Model] Extend Ultravox to accept audio longer than 30s (vllm-project…
80e78d0 
…#13631)

Signed-off-by: Farzad Abdolhosseini <[email protected]>
@aarnphm
[V1][Core] Support MistralTokenizer for Structured Output (vllm-proje…
77a318b 
…ct#14625)

Signed-off-by: Aaron Pham <[email protected]>
@Isotr0py
[Core] Refactor QKVCrossParallelLinear implementation to support BN…
e392d85 
…B 4-bit quantization (vllm-project#14545)

Signed-off-by: Isotr0py <[email protected]>
@SzymonOzog
[Kernel] GGUF MoE kernel (vllm-project#14613)
e22ee1e 
Signed-off-by: SzymonOzog <[email protected]>
@benchislett
[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative dec…
5c538c3 
…oding due to batch indexing (vllm-project#14645)

Signed-off-by: Benjamin Chislett <[email protected]>
@pavanimajety
[Kernel] Add ModelOpt FP4 Checkpoint Support (vllm-project#12520)
debd6bb 
Signed-off-by: Pavani Majety <[email protected]>
@bigPYJ1151 @Isotr0py
[CPU] Upgrade CPU backend to torch-2.6 (vllm-project#13381)
ff47aab 
Signed-off-by: jiang1.li <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
@SageMoore
[ROCm][Bugfix] Ensure that the moe_wna16_gemm kernel is not built on …
45f3f3f 
…ROCm platforms. (vllm-project#14629)

Signed-off-by: Sage Moore <[email protected]>
@WoosukKwon @ywang96 @DarkLight1337
[Model] Add support for Gemma 3 (vllm-project#14660)
c0c25e2 
Signed-off-by: Woosuk Kwon <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@ameyanjarlekar
[Bugfix] Missing thumbnail from NVLM-D processor (vllm-project#14633)
4a754fc 
Signed-off-by: ameyanjarlekar <[email protected]>
@SageMoore
[ROCm] Enable chunked prefill/paged attention in MLA on ROCm (vllm-pr…
d9f83d6 
…oject#14316)

Signed-off-by: Sage Moore <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main'
7be8c1f
@gshtras
Remove CMake FP8 conditioning
d7657c2
@gshtras gshtras merged commit b758abb into main Mar 12, 2025
9 of 13 checks passed
@gshtras gshtras deleted the unified_fp8 branch March 12, 2025 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

@Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD Awaiting requested review from Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD is a code owner

@maleksan85 maleksan85 Awaiting requested review from maleksan85 maleksan85 is a code owner

@sunway513 sunway513 Awaiting requested review from sunway513 sunway513 is a code owner

@divakar-amd divakar-amd Awaiting requested review from divakar-amd divakar-amd is a code owner

@charlifu charlifu Awaiting requested review from charlifu charlifu is a code owner

@mawong-amd mawong-amd Awaiting requested review from mawong-amd mawong-amd is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.