Parsing the with statement with tuple argument and type comment failed

[15r10nk]

Describe the bug

The following code can not be parsed/formatted by black:

with (name_5, something): # type: ignoresome text
    pass

(playground)

black reported the following error:

> black -l 100 -t py312 bug.py
error: cannot format bug.py: INTERNAL ERROR: Black 25.1.1.dev21+g944a38e.d20250317 on Python (CPython) 3
.12.6 produced code that is not equivalent to the source.  Please report a bug on https://github.com/psf
/black/issues.  This diff might be helpful: /tmp/blk_pgrn0rje.log

Oh no! 💥 💔 💥
1 file failed to reformat.

the reported diff in /tmp/blk_pgrn0rje.log is:

--- src
+++ dst
@@ -5,30 +5,29 @@
         Pass(
         )  # /Pass
         items=
         withitem(
             context_expr=
-            Tuple(
+            Name(
                 ctx=
                 Load(
                 )  # /Load
-                elts=
-                Name(
-                    ctx=
-                    Load(
-                    )  # /Load
-                    id=
-                    'name_5',  # str
-                )  # /Name
-                Name(
-                    ctx=
-                    Load(
-                    )  # /Load
-                    id=
-                    'something',  # str
-                )  # /Name
-            )  # /Tuple
+                id=
+                'name_5',  # str
+            )  # /Name
+            optional_vars=
+            None,  # NoneType
+        )  # /withitem
+        withitem(
+            context_expr=
+            Name(
+                ctx=
+                Load(
+                )  # /Load
+                id=
+                'something',  # str
+            )  # /Name
             optional_vars=
             None,  # NoneType
         )  # /withitem
         type_comment=
         'ignoresome text',  # str

but it can be parsed by cpython:

from ast import parse
parse(
    'with (name_5, something): # type: ignoresome text\n'
    '    pass\n'
)

The code can be formatted with black -l 100 -t py312 bug.py --fast:

with name_5, something:  # type: ignoresome text
    pass

Environment

• Black's version: current main (6144c46)
• OS and Python version: Linux/Python 3.12.6 (main, Sep 9 2024, 22:11:19) [Clang 18.1.8 ]

Additional context

The bug was found by pysource-codegen (see #3908)
The above problem description was created from a script, let me know if you think it can be improved.

[15r10nk]

this might be related to #4632

[tusharsadhwani]

Happy to report that the tool infact hit the minimal reproduction.

It can be changed cosmetically slightly:

with (x, y): # type:
    pass

But I'm very impressed.

[15r10nk]

And I'm happy to report that this is most likely the last bug the tool has found.

[tusharsadhwani]

I see the problem now, it is a deviation from expectation from CPython itself (at least from what I think)

In case of a type COMMENT, we get DIFFERENT ASTs, with and without the brackets:

>>> import ast
>>> x = '''
... with (x, y): # type: int
...   pass
... '''
>>> y = '''
... with x, y: # type: int
...   pass
... '''
>>> print(ast.dump(ast.parse(x, type_comments=True), indent=2))
Module(
  body=[
    With(
      items=[
        withitem(
          context_expr=Tuple(
            elts=[
              Name(id='x', ctx=Load()),
              Name(id='y', ctx=Load())],
            ctx=Load()))],
      body=[
        Pass()],
      type_comment='int')],
  type_ignores=[])
>>> print(ast.dump(ast.parse(y, type_comments=True), indent=2))
Module(
  body=[
    With(
      items=[
        withitem(
          context_expr=Name(id='x', ctx=Load())),
        withitem(
          context_expr=Name(id='y', ctx=Load()))],
      body=[
        Pass()],
      type_comment='int')],
  type_ignores=[])

In case of type IGNORE, we get IDENTICAL ASTs with or without bracket:

>>> x = '''
... with (x, y): # type: ignore
...   pass
... '''
>>> y = '''
... with x, y: # type: ignore
...   pass
... '''
>>> print(ast.dump(ast.parse(x, type_comments=True), indent=2))
Module(
  body=[
    With(
      items=[
        withitem(
          context_expr=Name(id='x', ctx=Load())),
        withitem(
          context_expr=Name(id='y', ctx=Load()))],
      body=[
        Pass()])],
  type_ignores=[
    TypeIgnore(lineno=2, tag='')])
>>> print(ast.dump(ast.parse(y, type_comments=True), indent=2))
Module(
  body=[
    With(
      items=[
        withitem(
          context_expr=Name(id='x', ctx=Load())),
        withitem(
          context_expr=Name(id='y', ctx=Load()))],
      body=[
        Pass()])],
  type_ignores=[
    TypeIgnore(lineno=2, tag='')])

[tusharsadhwani]

Honestly this feels like a CPython bug, type comment on a line doesn't change semantic meaning and therefore should not change the AST.

[15r10nk]

I agree. Do you want to create a cpython issue?

[tusharsadhwani]

Yup lemme do that.

[tusharsadhwani]

Created python/cpython#131570

[tusharsadhwani]

Update on this: This was fixed in Python 3.13, and the bug doesn't reproduce on Python 3.13 either. I think it's safe to close this, but I'll let @JelleZijlstra decide.

[15r10nk]

I think it would be ok to search the issues with pysource-codegen only with python 3.13, or to build some sort of white-list if needed.