Skip to content

Pandas>=3 breaking change in compare stage #335

@kpgbrock

Description

@kpgbrock

The latest pandas >= 3 moved to a dedicated string type (see documentation here) that seems to introduce an error in the compare stage in the develop branch, installed 4/3/26. From the failed file, the error appears to be happening at:

evcouplings/compare/sifts.py", line 836, in _create_mapping
n.loc[:, "j"] = n.j.astype(str)
~~~~~^^^^^^^^

which then goes on to call pandas/core/indexing.py, with the bottom-line error:

line 2163, in _setitem_single_column
raise TypeError(
f"Invalid value '{value}' for dtype '{dtype}'"
) from exc
TypeError: Invalid value '
[ '2', '3', '4', '5', '6', '7', '8', '9', '10', '11',
...
'228', '229', '230', '231', '232', '233', '234', '235', '236', '237']
Length: 236, dtype: str' for dtype 'int64'

I confirmed that reverting to pandas=2.3.3 did not throw the error, and was able to run through the mutate stage successfully. I didn't get a chance to troubleshoot with the new pandas installation though. Hope y'all are doing well!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions