TST(string dtype): Resolve xfail in groupby.test_size

[rhshadrach]

• closes #xxxx (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

groupby does inference on the group labels across the board.

df = pd.DataFrame({"a": [1, 1, 2], "b": [3, 4, 5]}, dtype="object")
gb = df.groupby("a")
result = gb.sum()
print(result.index.dtype)
# int64

While I agree long-term I'd prefer to preserve object dytpe, I do not think we should be changing this at this point.

[WillAyd]

Why doesn't it work if you just remove the dtype argument and let the constructor infer?

[rhshadrach]

Good question - this was introduced in #55627 but I do not see why if the values are string[pyarrow] that the result would be Int64.

cc @phofl

[jorisvandenbossche]

@rhshadrach the Int64 is for exp_dtype on the line below, not for the dtype of the Index being constructed on this line, so I am not entirely understanding your comment/question ?
(the construction of exp_dtype is not being touched in this PR)

[rhshadrach]

Ah, thanks! When the grouping column is StringDtype, we preserve this even when infer_string is False. In the groupby code, the uniques that go into creating the index is a string array. When the input is object dtype and infer_string=True, we do inference on the values and coerce to dtype str.

So in the object case we're doing inference, whereas in the non-object case we are not. It seems reasonable to me, thoughts?

[WillAyd]

Ah OK thanks for the explanation. I am not sure how I feel yet, but at first I wasn't expecting the action of grouping to perform any inference. Is that not a performance hit?

[jorisvandenbossche]

FWIW, if this is not easy to "fix" (avoid the inference), then I am personally fine with the current behaviour for now

[WillAyd]

Great - I think we are all leaning in that direction

[rhshadrach]

The line in question is

pandas/pandas/core/groupby/ops.py

Line 755 in e3b2de8

levels = [Index._with_infer(ping.uniques) for ping in self.groupings]

While it's been moved around recently, I believe that's long standing behavior. I can investigate what impact removing that (so just using standard Index init) would have, but seems independent.

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout 2.3.x
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 354b61f88bc0523d4bb9f3cfe1d6c12f9a3d6567

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #60711: TST(string dtype): Resolve xfail in groupby.test_size'

4. Push to a named branch:

git push YOURFORK 2.3.x:auto-backport-of-pr-60711-on-2.3.x

5. Create a PR against branch 2.3.x, I would have named this PR:

"Backport PR #60711 on branch 2.3.x (TST(string dtype): Resolve xfail in groupby.test_size)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

[WillAyd]

Manual backport in #60782