Test skip hyghiene: osx by h-vetinari · Pull Request #353 · conda-forge/pytorch-cpu-feedstock

h-vetinari · 2025-02-09T22:35:05Z

Try to see which skips we still need after 2.6; also, in #326, I've had to skip the torchinductor tests because they were segfaulting on osx (in contrast to 2.5), see b627641. That's not a good thing, so we should ideally fix that.

conda-forge-admin · 2025-02-09T22:36:41Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

ℹ️ The magma output has been superseded by libmagma-devel.
ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/19841630979. Examine the logs at this URL for more detail.}

…5.11.23.16.08.13 Other tools: - conda-build 25.11.0 - rattler-build 0.51.0 - rattler-build-conda-compat 1.4.10

h-vetinari · 2025-12-03T03:27:12Z

Something weird is going on with MPS on osx. Look at the timestamps vs. the percentages

2025-12-02T01:20:53.4025390Z TEST END: /Users/runnerx/miniforge3/conda-bld/osx-arm64/pytorch-cpu-2.9.1-cpu_generic_hcc7c195_0.conda
2025-12-02T01:20:53.4200610Z TEST START: /Users/runnerx/miniforge3/conda-bld/osx-arm64/pytorch-tests-2.9.1-cpu_generic_py313_ha249dfb_0.conda         # first `pytorch-tests` output being tested
2025-12-02T01:21:50.3558240Z [gw0] [  0%] PASSED [0.0048s] test/test_autograd.py::TestAutograd::test_access_saved_tensor_twice_without_recomputation_works 
2025-12-02T01:22:24.8448650Z [gw0] [ 10%] PASSED [0.0008s] test/test_custom_ops.py::TestGenerateOpcheckTests::test_generate_repro_no_save_data 
2025-12-02T01:22:31.8731700Z [gw0] [ 20%] PASSED [0.0027s] test/test_linalg.py::TestLinalgCPU::test_addmm_mv_transpose_a_True_transpose_b_False_alpha_1_0_beta_1_0_cpu_float32 
2025-12-02T01:23:25.2647630Z [gw0] [ 30%] PASSED [0.0010s] test/test_modules.py::TestModuleCPU::test_to_empty_nn_AvgPool1d_swap_True_cpu_float32 
2025-12-02T01:24:39.7525120Z [gw0] [ 40%] PASSED [0.0047s] test/test_modules.py::TestModuleMPS::test_factory_kwargs_nn_AvgPool2d_mps_float32 
2025-12-02T01:25:40.4699770Z [gw0] [ 50%] PASSED [0.0052s] test/test_modules.py::TestModuleMPS::test_factory_kwargs_nn_Softmin_mps_float16 
2025-12-02T01:39:27.8740250Z [gw0] [ 51%] PASSED [94.4943s] test/test_modules.py::TestModuleMPS::test_forward_nn_GRU_train_mode_mps_float16 
2025-12-02T02:27:05.3767080Z [gw1] [ 52%] PASSED [13.7502s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_GELU_mps_float16 
2025-12-02T03:56:01.5618180Z [gw1] [ 53%] PASSED [26.7351s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_LeakyReLU_mps_float16 
2025-12-02T04:10:42.2153250Z [gw0] [ 54%] PASSED [0.0066s] test/test_modules.py::TestModuleMPS::test_if_train_and_eval_modes_differ_nn_KLDivLoss_mps_float16 
2025-12-02T05:03:19.4796450Z [gw0] [ 55%] PASSED [4.2550s] test/test_modules.py::TestModuleMPS::test_memory_format_nn_AdaptiveMaxPool1d_mps_float16 
2025-12-02T05:29:56.3872950Z [gw0] [ 56%] PASSED [9.0542s] test/test_modules.py::TestModuleMPS::test_memory_format_nn_LeakyReLU_mps_float16 
2025-12-02T06:01:27.7234130Z [gw0] [ 57%] SKIPPED [0.0013s] test/test_modules.py::TestModuleMPS::test_multiple_device_transfer_nn_AvgPool1d_mps_float32 
2025-12-02T06:07:00.7160070Z [gw0] [ 58%] PASSED [332.8067s] test/test_modules.py::TestModuleMPS::test_to_nn_MultiheadAttention_eval_mode_swap_False_set_grad_True_mps_float32 
2025-12-02T07:27:17.9352830Z [gw0] [ 59%] PASSED [0.0012s] test/test_modules.py::TestModuleMPS::test_to_nn_Sigmoid_swap_False_set_grad_True_mps_float32 
2025-12-02T11:25:27.6268670Z [gw1] [ 60%] PASSED [0.0013s] test/test_modules.py::TestModuleMPS::test_repr_nn_ConstantPad2d_mps_float16 
# timeout

The entirety of the insane slowdown seems to happen in TestModuleMPS.

This group of test in particular looks like it's almost uniformly hitting the 20min timeout

Details

2025-12-02T07:27:18.5644790Z [gw0] [ 59%] SKIPPED [0.0018s] test/test_modules.py::TestModuleMPS::test_to_nn_Threshold_swap_True_set_grad_False_mps_float32 
2025-12-02T07:27:18.5673200Z [gw0] [ 59%] SKIPPED [0.0016s] test/test_modules.py::TestModuleMPS::test_to_nn_Threshold_swap_True_set_grad_True_mps_float32 
2025-12-02T07:34:22.6295920Z [gw1] [ 59%] FAILED [1200.2075s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerDecoderLayer_mps_float16 
2025-12-02T07:40:28.2223580Z [gw0] [ 59%] PASSED [789.6524s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerDecoderLayer_swap_False_set_grad_False_mps_float32 
2025-12-02T07:55:46.5797120Z [gw1] [ 59%] FAILED [1200.5276s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerDecoderLayer_mps_float32 
2025-12-02T08:00:20.1754060Z [gw0] [ 59%] PASSED [1191.9146s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerDecoderLayer_swap_False_set_grad_True_mps_float32 
2025-12-02T08:16:07.5761810Z [gw1] [ 59%] FAILED [1200.1559s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoderLayer_eval_mode_mps_float16 
2025-12-02T08:20:29.3480650Z [gw0] [ 59%] FAILED [1200.2860s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerDecoderLayer_swap_True_set_grad_False_mps_float32 
2025-12-02T08:37:08.6361450Z [gw1] [ 59%] FAILED [1200.4452s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoderLayer_eval_mode_mps_float32 
2025-12-02T08:40:17.3998890Z [gw0] [ 59%] PASSED [1188.0639s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerDecoderLayer_swap_True_set_grad_True_mps_float32 
2025-12-02T08:44:55.4171130Z [gw0] [ 59%] PASSED [278.0122s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_eval_mode_swap_False_set_grad_False_mps_float32 
2025-12-02T08:52:03.0410990Z [gw0] [ 59%] PASSED [427.6176s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_eval_mode_swap_False_set_grad_True_mps_float32 
2025-12-02T08:57:35.0610220Z [gw1] [ 59%] FAILED [1200.1559s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoderLayer_train_mode_mps_float16 
2025-12-02T08:59:02.0464050Z [gw0] [ 59%] PASSED [419.0094s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_eval_mode_swap_True_set_grad_False_mps_float32 
2025-12-02T09:05:58.5958150Z [gw0] [ 59%] PASSED [416.5351s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_eval_mode_swap_True_set_grad_True_mps_float32 
2025-12-02T09:10:56.9110220Z [gw0] [ 59%] PASSED [298.3005s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_train_mode_swap_False_set_grad_False_mps_float32 
2025-12-02T09:17:58.3751530Z [gw1] [ 59%] FAILED [1200.5472s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoderLayer_train_mode_mps_float32 
2025-12-02T09:18:44.1535940Z [gw0] [ 59%] PASSED [467.2481s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_train_mode_swap_False_set_grad_True_mps_float32 
2025-12-02T09:26:29.5150820Z [gw0] [ 59%] PASSED [465.3572s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_train_mode_swap_True_set_grad_False_mps_float32 
2025-12-02T09:34:08.8447110Z [gw0] [ 59%] PASSED [459.3250s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoderLayer_train_mode_swap_True_set_grad_True_mps_float32 
2025-12-02T09:38:43.6662340Z [gw0] [ 59%] PASSED [274.8093s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_eval_mode_swap_False_set_grad_False_mps_float32 
2025-12-02T09:39:11.4926780Z [gw1] [ 59%] FAILED [1200.3873s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoder_eval_mode_mps_float16 
2025-12-02T09:47:59.7064400Z [gw0] [ 59%] PASSED [556.0775s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_eval_mode_swap_False_set_grad_True_mps_float32 
2025-12-02T09:57:49.4321400Z [gw0] [ 59%] PASSED [589.7299s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_eval_mode_swap_True_set_grad_False_mps_float32 
2025-12-02T09:59:40.0763300Z [gw1] [ 59%] FAILED [1200.4937s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoder_eval_mode_mps_float32 
2025-12-02T10:07:03.4547830Z [gw0] [ 59%] PASSED [554.0258s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_eval_mode_swap_True_set_grad_True_mps_float32 
2025-12-02T10:12:12.9435740Z [gw0] [ 59%] PASSED [309.4906s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_train_mode_swap_False_set_grad_False_mps_float32 
2025-12-02T10:20:51.8442370Z [gw1] [ 59%] FAILED [1200.2544s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoder_train_mode_mps_float16 
2025-12-02T10:22:36.8392400Z [gw0] [ 59%] PASSED [625.8716s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_train_mode_swap_False_set_grad_True_mps_float32 
2025-12-02T10:33:17.9865430Z [gw0] [ 59%] PASSED [639.1128s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_train_mode_swap_True_set_grad_False_mps_float32 
2025-12-02T10:41:39.4753490Z [gw1] [ 59%] FAILED [1200.3611s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_TransformerEncoder_train_mode_mps_float32 
2025-12-02T10:43:40.8275870Z [gw0] [ 59%] PASSED [622.8461s] test/test_modules.py::TestModuleMPS::test_to_nn_TransformerEncoder_train_mode_swap_True_set_grad_True_mps_float32 
2025-12-02T11:01:40.0155790Z [gw1] [ 59%] FAILED [1200.4659s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_Transformer_mps_float16 
2025-12-02T11:03:40.9389770Z [gw0] [ 59%] FAILED [1200.0481s] test/test_modules.py::TestModuleMPS::test_to_nn_Transformer_swap_False_set_grad_False_mps_float32 
2025-12-02T11:22:26.3453670Z [gw1] [ 59%] FAILED [1200.2110s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_Transformer_mps_float32 
2025-12-02T11:22:41.6893080Z [gw1] [ 59%] PASSED [15.3437s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_ZeroPad1d_mps_float16 
2025-12-02T11:22:56.4771680Z [gw1] [ 59%] PASSED [14.7844s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_ZeroPad1d_mps_float32 
2025-12-02T11:23:12.0592870Z [gw1] [ 59%] PASSED [15.5815s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_ZeroPad2d_mps_float16 
2025-12-02T11:23:26.4765580Z [gw1] [ 59%] PASSED [14.4153s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_ZeroPad2d_mps_float32 
2025-12-02T11:23:40.3271920Z [gw1] [ 59%] PASSED [13.8485s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_ZeroPad3d_mps_float16 
2025-12-02T11:23:48.7002260Z [gw0] [ 59%] FAILED [1200.2521s] test/test_modules.py::TestModuleMPS::test_to_nn_Transformer_swap_False_set_grad_True_mps_float32 
2025-12-02T11:23:53.8750140Z [gw1] [ 59%] PASSED [13.5454s] test/test_modules.py::TestModuleMPS::test_non_contiguous_tensors_nn_ZeroPad3d_mps_float32 
2025-12-02T11:23:53.8827200Z [gw1] [ 59%] SKIPPED [0.0042s] test/test_modules.py::TestModuleMPS::test_repr_nn_AdaptiveAvgPool1d_mps_float16 
2025-12-02T11:23:53.8865440Z [gw1] [ 59%] SKIPPED [0.0031s] test/test_modules.py::TestModuleMPS::test_repr_nn_AdaptiveAvgPool1d_mps_float32 
2025-12-02T11:23:53.8899970Z [gw1] [ 59%] SKIPPED [0.0021s] test/test_modules.py::TestModuleMPS::test_repr_nn_AdaptiveAvgPool2d_mps_float16

mgorny · 2026-01-02T12:06:42Z

@h-vetinari, should I include these changes (and the win PR) if I do the next bump?

h-vetinari · 2026-01-02T12:49:54Z

You could use a filtered+squashed version of #352, thanks; this PR isn't ready yet because I started running into #462

h-vetinari mentioned this pull request Feb 11, 2025

pytorch v2.6.0 #326

Merged

h-vetinari added 4 commits November 25, 2025 15:10

re-enable torchinductor tests on osx; remove skips

577b637

DEBUG: only osx

da4043e

run tests verbosely

23b2031

MNT: Re-rendered with conda-smithy 3.53.3 and conda-forge-pinning 202…

622cbda

…5.11.23.16.08.13 Other tools: - conda-build 25.11.0 - rattler-build 0.51.0 - rattler-build-conda-compat 1.4.10

h-vetinari force-pushed the skips_osx branch from f30dce8 to 622cbda Compare November 25, 2025 04:15

h-vetinari added 12 commits November 25, 2025 17:04

re-add necessary skips

b12b37a

skip a hanging test and add a timeout to the test suite

e08a89b

another skip

77ee6af

more skips

dc26732

more skips

7a1d3e3

widen skips

1b9ce88

next skip

8d24538

more skips

d8491fb

more skips

5b0aeb6

more skips

3ddb185

avoid too-aggressive timeout

cd4084b

increase timeout again

8d1e7d1

h-vetinari mentioned this pull request Dec 3, 2025

Sudden huge runtime for MPS tests on osx-arm #462

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Test skip hyghiene: osx#353

Test skip hyghiene: osx#353
h-vetinari wants to merge 16 commits intoconda-forge:mainfrom
h-vetinari:skips_osx

h-vetinari commented Feb 9, 2025

Uh oh!

conda-forge-admin commented Feb 9, 2025 •

edited

Loading

Uh oh!

h-vetinari commented Dec 3, 2025

Uh oh!

mgorny commented Jan 2, 2026

Uh oh!

h-vetinari commented Jan 2, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

h-vetinari commented Feb 9, 2025

Uh oh!

conda-forge-admin commented Feb 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

h-vetinari commented Dec 3, 2025

Uh oh!

mgorny commented Jan 2, 2026

Uh oh!

h-vetinari commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

conda-forge-admin commented Feb 9, 2025 •

edited

Loading

h-vetinari commented Jan 2, 2026 •

edited

Loading