Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Marlin moe integration #266

Closed
wants to merge 48 commits into from
Closed

Marlin moe integration #266

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
15d0f20
Start working on linking the elements together
ElizaWszola May 17, 2024
60c097c
Runs to completion
ElizaWszola May 21, 2024
42750bc
Commit before moving things around in the kernel
ElizaWszola May 22, 2024
b312ad5
Fix shared memory issue
ElizaWszola May 23, 2024
289cc8c
Fused marlin moe in model executor
ElizaWszola May 24, 2024
f7b106e
Various small fixes
ElizaWszola May 24, 2024
6765bde
quantize outside fused_marlin_moe function
ElizaWszola May 24, 2024
0886f76
Test shapes
ElizaWszola May 27, 2024
8ee85dd
Better test, some passing
ElizaWszola May 28, 2024
cb4dbaa
Working expert chunks
ElizaWszola Jun 3, 2024
cd4410f
working
ElizaWszola Jun 3, 2024
e9ee483
Combined kernel with fused op, scores
ElizaWszola Jun 11, 2024
c839319
Debugging
ElizaWszola Jun 12, 2024
5ac5ba6
Rand for everything
ElizaWszola Jun 12, 2024
7a9f453
Working A copy
ElizaWszola Jun 13, 2024
c412ab8
cleanup
ElizaWszola Jun 14, 2024
2d4ac6a
Merge branch 'main' into marlin-moe-integration
ElizaWszola Jun 14, 2024
ffd5f64
Try to fix pybind11 builds after merge
ElizaWszola Jun 14, 2024
a22272a
Continue work on bindings
ElizaWszola Jun 17, 2024
d2cba06
impl
ElizaWszola Jun 17, 2024
4b5a9f1
typo fix
ElizaWszola Jun 17, 2024
0991c74
all links but maxdiff (must fix assertions)
ElizaWszola Jun 17, 2024
9dd97b5
it works
ElizaWszola Jun 17, 2024
c62bc7f
Merge branch 'main' into marlin-moe-integration
ElizaWszola Jun 17, 2024
e65c195
lots of debugging
ElizaWszola Jun 24, 2024
5f1ebdb
Fixed types, it's working!
ElizaWszola Jun 24, 2024
836d627
Cleanup
ElizaWszola Jun 25, 2024
d3665fa
Tiny cleanups
ElizaWszola Jun 25, 2024
4bcfde6
Some renaming
ElizaWszola Jun 25, 2024
7701ee7
Renaming, Bill's feedback
ElizaWszola Jun 25, 2024
b9fdda3
Format
ElizaWszola Jun 25, 2024
1b05843
add act_order to marlin moe
ElizaWszola Jun 26, 2024
8adcb26
Tensor constness, factor common fused_moe parts into smaller functions
ElizaWszola Jun 26, 2024
6a3ef46
Spelling
ElizaWszola Jun 26, 2024
c676bea
single marlin test should still be disabled
ElizaWszola Jun 27, 2024
53c8ff7
Pass unit tests
ElizaWszola Jul 1, 2024
0fd62c8
integrate with model
ElizaWszola Jul 2, 2024
458c83f
cleanups
ElizaWszola Jul 2, 2024
400e35a
Merge branch 'marlin-moe-act_order' into marlin-moe-integration
ElizaWszola Jul 2, 2024
a0d7f77
Merge branch 'main' into marlin-moe-integration
ElizaWszola Jul 4, 2024
f629593
format
ElizaWszola Jul 4, 2024
c8b79f3
Merge branch 'main' into marlin-moe-integration
ElizaWszola Jul 11, 2024
e405879
Start work on integrating with refactored mixtral code
ElizaWszola Jul 11, 2024
cda9a0f
Runs to completion, but produces garbage
ElizaWszola Jul 12, 2024
c469b74
it works!
ElizaWszola Jul 12, 2024
d8b455f
Cleanup, format, minor fixes
ElizaWszola Jul 12, 2024
7504696
more efficient m blocking, a couple small fixes
ElizaWszola Jul 23, 2024
3641692
Multi-GPU works, but could make it faster
ElizaWszola Jul 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,8 @@ define_gpu_extension_target(

set(VLLM_MOE_EXT_SRC
"csrc/moe/torch_bindings.cpp"
"csrc/moe/topk_softmax_kernels.cu")
"csrc/moe/topk_softmax_kernels.cu"
"csrc/moe/marlin_moe_ops.cu")

define_gpu_extension_target(
_moe_C
Expand Down
Loading
Loading