Skip to content

[Performance] Enable prefetching for tt.load with tensor of pointer #3484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
chengjunlu opened this issue Feb 21, 2025 · 2 comments · Fixed by #4076, #4064, #4016 or #4030 · May be fixed by #3634
Closed

[Performance] Enable prefetching for tt.load with tensor of pointer #3484

chengjunlu opened this issue Feb 21, 2025 · 2 comments · Fixed by #4076, #4064, #4016 or #4030 · May be fixed by #3634
Assignees
@etiotto
Labels
performance
Milestone
4. [Performance] ...

Comments

@chengjunlu
Copy link
Contributor

Enable prefetching for tt.load with tensor of pointer
@chengjunlu chengjunlu self-assigned this Feb 21, 2025
@vlad-penkin vlad-penkin added this to the 4.0 [Performance] Core milestone Feb 24, 2025
@chengjunlu chengjunlu linked a pull request Mar 10, 2025 that will close this issue
[Performance] Add the support of tensor of pointer in the prefetching and loop pipelining #3634
Draft
@etiotto etiotto self-assigned this Apr 28, 2025
@etiotto
Copy link
Contributor

etiotto commented Apr 28, 2025

I can help with this PR.

This was linked to pull requests May 2, 2025
Remove option support-regular-ptr #4076
Merged
[prefetch]: Add support for prefetching load operations using a tensor of pointers #4064
Merged
[MatmulLoopPipeline] Predicate PrefetchOp #4016
Merged
[MatmulLoopPipeline] Populate LoadOp mask to PrefetchOp #4030
Merged
@etiotto
Copy link
Contributor

etiotto commented May 5, 2025

Currently there is one last PR to merge. With that and previous work already merged the performance of GEMM using tensor of pointers improves by almost 100% as seen in the following graph:

Image

@etiotto etiotto reopened this May 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@etiotto etiotto

Labels
performance
Projects
None yet
Milestone
4. [Performance] Core
4 participants
@whitneywhtsang @etiotto @chengjunlu @vlad-penkin