Run Windows CI across all supported Python versions

[ichard26]

Let's see how slow or not slow this is...

This still needs updates to the development documentation.

[ichard26]

They're on par with the rest of the CI jobs. This should be okay to deploy. PTAL.

[sbidoul]

My only reservation would be that the probability of failure in those jobs (that would not also be revealed by the linux jobs) is so low that it may not be worth the additional heating of the planet.

[notatallshaw]

My only reservation would be that the probability of failure in those jobs (that would not also be revealed by the linux jobs) is so low that it may not be worth the additional heating of the planet.

Following this logic, shouldn't everything that isn't latest/oldest be turned off other than macos-latest? Which seems to be the fastest of all the CI jobs.

Then all the other Python versions can be run on a schedule, like in #13014.

My only strong opinion here is that all versions of Python for all supported OSes should be tested before a release. OS-Python version specific issues do come up with pip.

[notatallshaw]

I'm going to (cheekly) put this on to the 25.0 milestone, so this approach can be accepted or rejected by then, discussion was here: #13011

[sbidoul]

OS-Python version specific issues do come up with pip.

Do we know a case where, say, tests pass on all platforms on python 3.8 and 3.13, but fail on an intermediate version only on a specific platform ?

If not I would not do this for the reason in #13016 (comment)

[ichard26]

Do we know a case where, say, tests pass on all platforms on python 3.8 and 3.13, but fail on an intermediate version only on a specific platform ?

We've had at least one case of that but I can't find it (although IIRC it was a problem with the actions/setup-python action as one of the Windows job were failing with path too long errors despite long paths being enabled). I can't find anything else using GitHub search.

I would be fine with just running the full Windows CI on a weekly basis. I don't feel particularly strongly about this.

[sbidoul]

Ok, I'm removing this from the release milestone since this can be done independently of any release.

[ichard26]

Sounds good :)

[pfmoore]

I don't see any urgency to change things here. I don't think this is likely to make a significant difference to how many bugs we catch with CI, and while we're not in the same league as AI training when it comes to compute usage, keeping our power expenditure even a little lower seems worthwhile.