🐼 Upgrade PUDL to Pandas 3.0

[zaneselvans]

Overview

We should be able to upgrade to Pandas v3.0 during the summer of 2024 2026! This is a major release with lots of API and behavior changes and we use Pandas a ton, so we should expect breakage. Blog post with just the highlights.

• I have been working with the clankers 🤖 to migrate PUDL to Pandas 3.0 with as few behavioral changes as possible in 🐼 WIP: Upgrade PUDL to Pandas 3.0 #5130
• Unsurprisingly, this process highlighted a bunch of issues that weren't strictly dependent on Pandas 3.0, but that are related and often introduce behavior changes when addressed.
• Rather than mix all of those changes in with the PR that literally transitions PUDL from Pandas 2 to 3, I started PR 🐼 WIP: Prepare PUDL for Pandas 3.0 #5607 as a place to accumulate the bugfixes, so that they could merge into main first, allowing us to limit 🐼 WIP: Upgrade PUDL to Pandas 3.0 #5130 to only the changes required to switch from Pandas 2 to 3.
• Too many fixes accumulated on that branch for it to be easily reviewable, so I decided to split them each out into their own small PRs with narrow, dedicated issues. I've added them all as sub-issues here so we can discuss them separately and hopefully each one is easy to understand.
• I'm also using the Pandas 3.0 migration as a test case for constraining programmatic refactoring by comparing PUDL data outputs. Either ensuring that changes which shouldn't have any impact on the outputs in fact don't, or when impacts are expected, that we can see what they are and try to understand why things changed.

[e-belfer]

It's out! https://pandas.pydata.org/docs/dev/whatsnew/v3.0.0.html

This should fix a weird NaN dtype issue that arose in the latest quarterly update: #4993

[zaneselvans]

Should also get rid of all of our defensive df.copy() calls, lots of dtype issues, and... I'm sure it'll break other things. Excited to get it updated though!