Fix tensor dimensions for Jython 3 #261

khatchad · 2025-06-13T18:28:12Z

Fix #42 and fix #147 and fix #165 and fix #164 and fix #166.

Add logging and comments.
Add workarounds for Tensor type propagation stops upon tensor shape mismatch #195.
The new API being called from Jython 3 now needs new dependencies. See Jython3 installation is fragile wrt dependencies #258.

wala#42.

Some of the workarounds for wala#42 have now become workarounds for wala#195.

codecov · 2025-06-13T19:07:23Z

Codecov Report

Attention: Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.

Project coverage is 58.56%. Comparing base (a9a73d7) to head (2903750).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...com/ibm/wala/cast/python/loader/Python3Loader.java	91.66%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #261      +/-   ##
============================================
+ Coverage     55.01%   58.56%   +3.55%     
- Complexity      502      517      +15     
============================================
  Files           109      109              
  Lines          7663     7265     -398     
  Branches        827      827              
============================================
+ Hits           4216     4255      +39     
+ Misses         3210     2834     -376     
+ Partials        237      176      -61

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

khatchad · 2025-06-13T19:23:23Z

...m.wala.cast.python.ml.test/source/com/ibm/wala/cast/python/ml/test/TestTensorflow2Model.java

  public void testDecoratedMethod2() throws ClassHierarchyException, CancelException, IOException {
-    // NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.


Instead of 0,0, we are expecting an assertion error.

khatchad · 2025-06-13T19:23:41Z

...m.wala.cast.python.ml.test/source/com/ibm/wala/cast/python/ml/test/TestTensorflow2Model.java

  public void testDecoratedMethod3() throws ClassHierarchyException, CancelException, IOException {
-    // NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.


Instead of 0,0, we are expecting an assertion error.

khatchad · 2025-06-13T19:24:11Z

...m.wala.cast.python.ml.test/source/com/ibm/wala/cast/python/ml/test/TestTensorflow2Model.java

  public void testDecoratedMethod10() throws ClassHierarchyException, CancelException, IOException {
-    // NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.


Instead of 0,0, we are expecting an assertion error.

khatchad · 2025-06-13T19:24:37Z

...m.wala.cast.python.ml.test/source/com/ibm/wala/cast/python/ml/test/TestTensorflow2Model.java

  public void testDecoratedMethod13() throws ClassHierarchyException, CancelException, IOException {
-    // NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.


Instead of 0,0, we are expecting an assertion error.

msridhar · 2025-06-25T00:32:17Z

@khatchad lost track of this one is it still waiting on a review?

khatchad · 2025-06-25T04:09:54Z

@khatchad lost track of this one is it still waiting on a review?

@msridhar Yes. Thanks!

Copilot

Pull Request Overview

This PR updates the project to support Jython 3 by fixing tensor dimension handling, adding new dependencies and logging, and introducing workarounds for open issues.

Added required test dependencies (asm-all, jnr-constants)
Renamed Jython modules/artifacts to use jython3
Enhanced Python3Loader with logging and syntax‐error handling; simplified Python3Interpreter initialization

Reviewed Changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
com.ibm.wala.cast.python.test/pom.xml	Added dependencies for ASM and JNR constants
com.ibm.wala.cast.python.ml/pom.xml	Renamed `jython` dependency to `jython3`
com.ibm.wala.cast.python.ml.test/source/com/ibm/wala/cast/python/ml/test/TestNeuroImageExamples.java	Contains unresolved merge conflict markers
com.ibm.wala.cast.python.ml.test/pom.xml	Updated test artifact to `jython3.test`
com.ibm.wala.cast.python.jython3/source/com/ibm/wala/cast/python/util/Python3Interpreter.java	Removed broad exception catch; lazy‐init now unguarded
com.ibm.wala.cast.python.jython3/source/com/ibm/wala/cast/python/loader/Python3Loader.java	Added logger, syntax‐error catch, and info‐level tracing

Comments suppressed due to low confidence (3)

com.ibm.wala.cast.python.ml.test/source/com/ibm/wala/cast/python/ml/test/TestNeuroImageExamples.java:24

Unresolved merge conflict markers are present here. Please remove the conflict regions and ensure the intended tensor-dimension logic is finalized.

<<<<<<< contrib_jython3_tensor_dims

com.ibm.wala.cast.python.jython3/source/com/ibm/wala/cast/python/util/Python3Interpreter.java:10

Lazy initialization of the static interp is not thread-safe. Consider synchronizing this method or using a thread-safe lazy holder to prevent race conditions.

    if (interp == null) {

com.ibm.wala.cast.python.jython3/source/com/ibm/wala/cast/python/loader/Python3Loader.java:105

[nitpick] Frequent logging in tight loops may degrade performance. Consider adjusting the log level or guarding this call if it's not needed in production.

                logger.info(() -> "Evaluating: " + s);

Copilot · 2025-06-25T20:46:33Z

com.ibm.wala.cast.python.test/pom.xml

@@ -51,6 +51,14 @@
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
    </dependency>
+    <dependency>
+      <groupId>org.ow2.asm</groupId>
+      <artifactId>asm-all</artifactId>


This new dependency doesn't specify a <version>. Please add a version or ensure it's managed by a parent POM to avoid build instability.

Suggested change

<artifactId>asm-all</artifactId>

<artifactId>asm-all</artifactId>

<version>5.2</version>

msridhar

One minor nitpick

msridhar · 2025-06-28T05:53:30Z

com.ibm.wala.cast.python.jython3/source/com/ibm/wala/cast/python/loader/Python3Loader.java

@@ -94,15 +101,25 @@ public ConstantFoldingRewriter createCAstRewriter(CAst ast) {
            return new ConstantFoldingRewriter(ast) {
              @Override
              protected Object eval(CAstOperator op, Object lhs, Object rhs) {
+                String s = lhs + " " + op.getValue() + " " + rhs;


You could declare s inside the lambda body so it's only computed if logging is enabled

khatchad added 3 commits June 13, 2025 14:24

Progress on tensor shape resolution for Python 3.

1efda32

wala#42.

Workaround wala#195.

028d279

Some of the workarounds for wala#42 have now become workarounds for wala#195.

Add logging and comments.

d23ba6b

khatchad self-assigned this Jun 13, 2025

khatchad added bug Something isn't working parsing jython labels Jun 13, 2025

Fix wala#147.

06a5053

khatchad requested review from msridhar and juliandolby June 13, 2025 19:06

khatchad marked this pull request as ready for review June 13, 2025 19:06

khatchad requested a review from Copilot June 13, 2025 19:06

This comment was marked as outdated.

Sign in to view

khatchad enabled auto-merge (squash) June 13, 2025 19:10

khatchad mentioned this pull request Jun 13, 2025

Point example back to main #257

Open

khatchad disabled auto-merge June 13, 2025 19:18

khatchad enabled auto-merge (squash) June 13, 2025 19:18

khatchad commented Jun 13, 2025

View reviewed changes

khatchad added 2 commits June 13, 2025 17:41

Merge branch 'master' into contrib_jython3_tensor_dims

239ac7f

Merge branch 'master' into contrib_jython3_tensor_dims

7e6b7f5

Merge branch 'master' into contrib_jython3_tensor_dims

efd8c6c

Merge branch 'master' into contrib_jython3_tensor_dims

e7de0a6

khatchad requested a review from Copilot June 25, 2025 20:45

Copilot AI reviewed Jun 25, 2025

View reviewed changes

Fix merge conflict problem.

edf7400

msridhar approved these changes Jun 28, 2025

View reviewed changes

msridhar disabled auto-merge June 28, 2025 05:54

Merge branch 'master' into contrib_jython3_tensor_dims

2903750

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix tensor dimensions for Jython 3 #261

Fix tensor dimensions for Jython 3 #261

Uh oh!

khatchad commented Jun 13, 2025 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Jun 13, 2025 •

edited

Loading

Uh oh!

khatchad Jun 13, 2025

Uh oh!

khatchad Jun 13, 2025

Uh oh!

khatchad Jun 13, 2025

Uh oh!

khatchad Jun 13, 2025

Uh oh!

msridhar commented Jun 25, 2025

Uh oh!

khatchad commented Jun 25, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jun 25, 2025

Uh oh!

msridhar left a comment

Uh oh!

msridhar Jun 28, 2025

Uh oh!

Uh oh!

		public void testDecoratedMethod2() throws ClassHierarchyException, CancelException, IOException {
		// NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.

		public void testDecoratedMethod3() throws ClassHierarchyException, CancelException, IOException {
		// NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.

		public void testDecoratedMethod10() throws ClassHierarchyException, CancelException, IOException {
		// NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.

		public void testDecoratedMethod13() throws ClassHierarchyException, CancelException, IOException {
		// NOTE: Change to 0, 0 once https://github.com/wala/ML/issues/147 is fixed.

	<artifactId>asm-all</artifactId>
	<artifactId>asm-all</artifactId>
	<version>5.2</version>

Fix tensor dimensions for Jython 3 #261

Are you sure you want to change the base?

Fix tensor dimensions for Jython 3 #261

Uh oh!

Conversation

khatchad commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

khatchad Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

khatchad Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

khatchad Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

khatchad Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

msridhar commented Jun 25, 2025

Uh oh!

khatchad commented Jun 25, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

msridhar left a comment

Choose a reason for hiding this comment

Uh oh!

msridhar Jun 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

khatchad commented Jun 13, 2025 •

edited

Loading

codecov bot commented Jun 13, 2025 •

edited

Loading