Skip to content

[MatmulLoopPipeline] Predicate PrefetchOp #4016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 26, 2025

Conversation

whitneywhtsang
Copy link
Contributor

@whitneywhtsang whitneywhtsang commented Apr 25, 2025

Now that PrefetchOp takes mask as an argument, we can handle predication of PrefetchOp in MatmulLoopPipeline.
Benchmark CI: https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/14657236822 (No performance regressions.)

Note: this change comes partially from #3634.

@whitneywhtsang whitneywhtsang self-assigned this Apr 25, 2025
@@ -154,7 +155,7 @@ static Value getPredMask(RewriterBase &rewriter, Type typeLike,
Value currentMask, Value pred) {
Location loc = pred.getLoc();
Value mask = pred;
Type maskType = tt::getI1SameShape(typeLike);
Type maskType = tt::getI1SameShape(tt::getPointeeType(typeLike));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type of mask should be getI1SameShape(getPointeeType($_self)) as defined in https://github.com/intel/intel-xpu-backend-for-triton/blob/main/include/triton/Dialect/Triton/IR/TritonOps.td#L233.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does typeLike means ? Lets document the parameters this function take and also what it does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added function description.

@whitneywhtsang whitneywhtsang force-pushed the whitneywhtsang/prefetch_pipeline branch from 3fd9a80 to 6c7a802 Compare April 25, 2025 05:02
@@ -154,7 +155,7 @@ static Value getPredMask(RewriterBase &rewriter, Type typeLike,
Value currentMask, Value pred) {
Location loc = pred.getLoc();
Value mask = pred;
Type maskType = tt::getI1SameShape(typeLike);
Type maskType = tt::getI1SameShape(tt::getPointeeType(typeLike));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does typeLike means ? Lets document the parameters this function take and also what it does.

@whitneywhtsang whitneywhtsang force-pushed the whitneywhtsang/prefetch_pipeline branch 2 times, most recently from 88fe8be to a4b2e2a Compare April 25, 2025 15:15
Signed-off-by: Whitney Tsang <[email protected]>
@whitneywhtsang whitneywhtsang force-pushed the whitneywhtsang/prefetch_pipeline branch from a4b2e2a to cc4150d Compare April 25, 2025 15:19
@whitneywhtsang whitneywhtsang marked this pull request as ready for review April 25, 2025 15:19
@whitneywhtsang whitneywhtsang requested review from alexbaden and a team April 25, 2025 15:20
Base automatically changed from whitneywhtsang/prefetch_mask to main April 25, 2025 15:25
@whitneywhtsang whitneywhtsang requested a review from etiotto April 25, 2025 15:26
@whitneywhtsang whitneywhtsang enabled auto-merge (squash) April 26, 2025 19:13
@whitneywhtsang whitneywhtsang merged commit 8aba8cb into main Apr 26, 2025
14 of 16 checks passed
@whitneywhtsang whitneywhtsang deleted the whitneywhtsang/prefetch_pipeline branch April 26, 2025 21:49
@etiotto etiotto linked an issue May 2, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Performance] Enable prefetching for tt.load with tensor of pointer
4 participants