add PytatoKeyBuilder, persistent_dict test #459

matthiasdiener · 2023-09-25T22:37:20Z

Alternative to #457.

See #547 for a more in-depth discussion.

Please squash

matthiasdiener · 2023-09-25T22:42:27Z

pytato/analysis/__init__.py

+    def update_for_pymbolic_expression(self, key_hash: Any, key: Any) -> None:
+        if key is None:
+            self.update_for_NoneType(key_hash, key)  # type: ignore[no-untyped-call]
+        else:
+            PersistentHashWalkMapper(key_hash)(key)
+
+    update_for_Product = update_for_pymbolic_expression  # noqa: N815
+    update_for_Sum = update_for_pymbolic_expression  # noqa: N815
+    update_for_If = update_for_pymbolic_expression  # noqa: N815
+    update_for_LogicalOr = update_for_pymbolic_expression  # noqa: N815
+    update_for_Call = update_for_pymbolic_expression  # noqa: N815
+    update_for_Comparison = update_for_pymbolic_expression  # noqa: N815
+    update_for_Quotient = update_for_pymbolic_expression  # noqa: N815
+    update_for_Power = update_for_pymbolic_expression  # noqa: N815


This could probably be improved, but I'm not sure how.

Perhaps inducer/pymbolic#125 resolves it.

I'm guessing this PR is waiting for inducer/pymbolic#125 ?

Yep, inducer/pymbolic#125 has resolved this, this bit should no longer be necessary. 😁

Removed in 70a6887

pytato/analysis/__init__.py

matthiasdiener · 2024-02-05T19:23:32Z

This is ready for review @inducer. The current version is able to hash the DAG of a 2-rank smoke_test_ks_3d run.

inducer · 2024-09-21T20:37:15Z

See also #547.

inducer · 2024-09-09T18:28:32Z

pytato/analysis/__init__.py

+        # CL Array
+        self.rec(key_hash, key.get())


Could you use an isinstance check to make sure that this only hashes the intended types?

Added in 70a6887

inducer · 2024-10-03T13:50:19Z

pytato/analysis/__init__.py

+    def update_for_pymbolic_expression(self, key_hash: Any, key: Any) -> None:
+        if key is None:
+            self.update_for_NoneType(key_hash, key)  # type: ignore[no-untyped-call]
+        else:
+            PersistentHashWalkMapper(key_hash)(key)
+
+    update_for_Product = update_for_pymbolic_expression  # noqa: N815
+    update_for_Sum = update_for_pymbolic_expression  # noqa: N815
+    update_for_If = update_for_pymbolic_expression  # noqa: N815
+    update_for_LogicalOr = update_for_pymbolic_expression  # noqa: N815
+    update_for_Call = update_for_pymbolic_expression  # noqa: N815
+    update_for_Comparison = update_for_pymbolic_expression  # noqa: N815
+    update_for_Quotient = update_for_pymbolic_expression  # noqa: N815
+    update_for_Power = update_for_pymbolic_expression  # noqa: N815


Yep, inducer/pymbolic#125 has resolved this, this bit should no longer be necessary. 😁

inducer · 2024-10-09T14:27:14Z

pytato/analysis/__init__.py

+    update_for_BitwiseAnd = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_BitwiseNot = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_BitwiseOr = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_BitwiseXor = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Call = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_CallWithKwargs = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Comparison = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_If = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_FloorDiv = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_LeftShift = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_LogicalAnd = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_LogicalNot = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_LogicalOr = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Lookup = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Power = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Product = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Quotient = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Remainder = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_RightShift = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Subscript = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Sum = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815
+    update_for_Variable = LoopyKeyBuilder.update_for_pymbolic_expression  # noqa: N815


All this can go now that inducer/pymbolic#125 is in.

Removed in 70a6887

matthiasdiener · 2024-11-08T18:26:38Z

This is ready for another review @inducer .

inducer · 2024-12-04T16:51:18Z

Thanks!

matthiasdiener self-assigned this Sep 25, 2023

matthiasdiener force-pushed the PytatoKeyBuilder branch 2 times, most recently from b2d71a3 to caa5a75 Compare September 25, 2023 22:41

matthiasdiener commented Sep 25, 2023

View reviewed changes

matthiasdiener changed the title ~~add PytatoKeyBuilder~~ add PytatoKeyBuilder, persistent_dict test Sep 25, 2023

matthiasdiener requested a review from inducer September 26, 2023 15:17

matthiasdiener force-pushed the PytatoKeyBuilder branch 2 times, most recently from 72e5569 to b31657f Compare November 7, 2023 16:48

matthiasdiener force-pushed the PytatoKeyBuilder branch from b31657f to 88989de Compare November 16, 2023 19:58

matthiasdiener force-pushed the PytatoKeyBuilder branch from 88989de to 78003e1 Compare December 8, 2023 02:50

matthiasdiener force-pushed the PytatoKeyBuilder branch from 78003e1 to cd167af Compare January 25, 2024 20:40

matthiasdiener marked this pull request as ready for review January 26, 2024 15:23

matthiasdiener force-pushed the PytatoKeyBuilder branch 2 times, most recently from e9c2855 to 3fd38a1 Compare February 5, 2024 21:10

inducer force-pushed the PytatoKeyBuilder branch from 6238242 to 8fc6620 Compare February 6, 2024 23:06

matthiasdiener mentioned this pull request Feb 15, 2024

make find_distributed_partition deterministic #465

Closed

2 tasks

matthiasdiener force-pushed the PytatoKeyBuilder branch from 8fc6620 to 51aeba0 Compare March 19, 2024 15:26

inducer force-pushed the PytatoKeyBuilder branch from 51aeba0 to 0687a4c Compare April 1, 2024 18:26

matthiasdiener force-pushed the PytatoKeyBuilder branch from 0687a4c to 0a2e7be Compare May 15, 2024 16:30

matthiasdiener added 10 commits May 15, 2024 13:23

add PytatoKeyBuilder

38e4332

mypy fixes

4dd3250

support TaggableCLArray, Subscript

970e7bb

CL Array, function

95dec09

add prim.Variable

2ac10ee

fixes to ndarray, pymb expressions

62a13ae

flake8

b8e04bf

improve test

ad9aa28

add full invocation test

60d8e41

lint fixes

9d45e65

matthiasdiener added 7 commits July 3, 2024 15:57

Merge branch 'main' into PytatoKeyBuilder

b1aaa97

Merge branch 'main' into PytatoKeyBuilder

16516ec

Merge branch 'main' into PytatoKeyBuilder

454f273

Merge branch 'main' into PytatoKeyBuilder

993dfe4

lint

82a5f25

Merge branch 'main' into PytatoKeyBuilder

dc53746

Merge branch 'main' into PytatoKeyBuilder

97df5d7

Merge branch 'main' into PytatoKeyBuilder

93abdc1

inducer reviewed Oct 3, 2024

View reviewed changes

Merge branch 'main' into PytatoKeyBuilder

b35d841

inducer reviewed Oct 9, 2024

View reviewed changes

matthiasdiener added 2 commits October 9, 2024 09:33

add typecheck, remove pymbolic handling

70a6887

lint

4f9f95f

matthiasdiener mentioned this pull request Oct 9, 2024

add get_hash, make DAGs deterministic #457

Closed

pylint

cffda36

matthiasdiener force-pushed the PytatoKeyBuilder branch from fa5e716 to cffda36 Compare October 9, 2024 15:05

matthiasdiener added 3 commits November 7, 2024 14:01

Merge branch 'main' into PytatoKeyBuilder

47a31d6

Merge branch 'main' into PytatoKeyBuilder

57908b1

add arraycontext package for mypy

adfe6b2

matthiasdiener requested a review from inducer November 8, 2024 18:26

matthiasdiener and others added 3 commits November 14, 2024 19:39

Merge branch 'main' into PytatoKeyBuilder

20ed0d2

Merge branch 'main' into PytatoKeyBuilder

29c1833

Merge branch 'main' into PytatoKeyBuilder

8e9fe8a

inducer enabled auto-merge (squash) December 4, 2024 16:51

inducer merged commit 006510c into main Dec 4, 2024
11 checks passed

inducer deleted the PytatoKeyBuilder branch December 4, 2024 17:13

matthiasdiener mentioned this pull request Dec 4, 2024

Persistent hashing for DAGs #547

Closed

add PytatoKeyBuilder, persistent_dict test #459

add PytatoKeyBuilder, persistent_dict test #459

Uh oh!

Conversation

matthiasdiener commented Sep 25, 2023 • edited by inducer Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

matthiasdiener commented Feb 5, 2024

Uh oh!

inducer commented Sep 21, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthiasdiener commented Nov 8, 2024

Uh oh!

inducer commented Dec 4, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

matthiasdiener commented Sep 25, 2023 •

edited by inducer

Loading