bioconductor-skeleton fails on HTTP 302 redirects to packages

[snowpong]

Running bioconda-utils bioconductor-skeleton -b 3.20 recipes config.yml IlluminaHumanMethylationEPICv2anno.20a1.hg38 fails even though the package exists, because we get a 302 redirect, but it's not followed.

(/mnt/scratch/espenr/dev/bioconda-recipes/env) espenr@hepp:~/dev/bioconda-recipes$ bioconda-utils bioconductor-skeleton -b 3.20 recipes config.yml IlluminaHumanMethylationEPICv2anno.20a1.hg38
/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/bioconda_utils/utils.py:43: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resourc
es.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
14:11:58 URLLIB3 DEBUG Starting new HTTPS connection (1): bioconductor.org:443
14:11:58 URLLIB3 DEBUG https://bioconductor.org:443 "GET /packages/3.20/bioc/VIEWS HTTP/1.1" 200 4855510
14:12:01 URLLIB3 DEBUG Starting new HTTPS connection (1): bioconductor.org:443
14:12:02 URLLIB3 DEBUG https://bioconductor.org:443 "GET /packages/3.20/data/annotation/VIEWS HTTP/1.1" 200 788334
14:12:03 URLLIB3 DEBUG Starting new HTTPS connection (1): bioconductor.org:443
14:12:03 URLLIB3 DEBUG https://bioconductor.org:443 "GET /packages/3.20/data/experiment/VIEWS HTTP/1.1" 200 564434
14:12:03 URLLIB3 DEBUG Starting new HTTPS connection (1): bioconductor.org:443
14:12:03 URLLIB3 DEBUG https://bioconductor.org:443 "GET /packages/3.20/data/annotation/html/IlluminaHumanMethylationEPICv2anno.20a1.hg38.html HTTP/1.1" 200 5328
14:12:03 BIOCONDA INFO Making recipe for: IlluminaHumanMethylationEPICv2anno.20a1.hg38
14:12:03 BIOCONDA DEBUG IlluminaHumanMethylationEPICv2anno.20a1.hg38==1.0.0, BioC==3.20
14:12:03 URLLIB3 DEBUG Starting new HTTPS connection (1): bioconductor.org:443
14:12:04 URLLIB3 DEBUG https://bioconductor.org:443 "HEAD /packages/3.20/data/annotation/src/contrib/IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz HTTP/1.1" 302 0
14:12:04 URLLIB3 DEBUG Starting new HTTPS connection (1): bioarchive.galaxyproject.org:443
14:12:04 URLLIB3 DEBUG https://bioarchive.galaxyproject.org:443 "HEAD /IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz HTTP/1.1" 404 0
14:12:04 URLLIB3 DEBUG Starting new HTTPS connection (1): depot.galaxyproject.org:443
14:12:04 URLLIB3 DEBUG https://depot.galaxyproject.org:443 "HEAD /software/bioconductor-illuminahumanmethylationepicv2anno.20a1.hg38/bioconductor-illuminahumanmethylationepicv2anno.20a1.hg38_1.0.0_src_all.t
ar.gz HTTP/1.1" 404 0
Traceback (most recent call last):
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/bin/bioconda-utils", line 10, in <module>
    sys.exit(main())
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/bioconda_utils/cli.py", line 1124, in main
    argh.dispatch_commands([
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/argh/dispatching.py", line 511, in dispatch_commands
    dispatch(parser, *args, **kwargs)
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/argh/dispatching.py", line 199, in dispatch
    return run_endpoint_function(
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/argh/dispatching.py", line 270, in run_endpoint_function
    return _process_command_output(lines, output_file, raw_output, always_flush)
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/argh/dispatching.py", line 290, in _process_command_output
    for line in lines:
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/argh/dispatching.py", line 415, in _execute_command
    for line in result:
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/argh/dispatching.py", line 395, in _call
    result = function(*positional_values, **values_by_name)
  File "<boltons.funcutils.FunctionBuilder-17>", line 2, in bioconductor_skeleton
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/bioconda_utils/cli.py", line 137, in wrapper
    func(*args, **kwargs)
  File "<boltons.funcutils.FunctionBuilder-16>", line 2, in bioconductor_skeleton
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/bioconda_utils/cli.py", line 66, in wrapper
    func(*args, **kwargs)
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/bioconda_utils/cli.py", line 832, in bioconductor_skeleton
    _bioconductor_skeleton.write_recipe(
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/bioconda_utils/bioconductor_skeleton.py", line 1233, in write_recipe
    logger.info('Using tarball from %s', proj.tarball_url)
  File "/mnt/scratch/espenr/dev/bioconda-recipes/env/lib/python3.10/site-packages/bioconda_utils/bioconductor_skeleton.py", line 524, in tarball_url
    'Tried the following:\n\t' + '\n\t'.join(urls),
TypeError: sequence item 0: expected str instance, NoneType found

If you test with wget you'll see it gets a 302 and then follows this and downloads the file:

espenr@hepp:~/temp$ wget https://bioconductor.org/packages/3.20/data/annotation/src/contrib/IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz
--2025-06-24 14:49:20--  https://bioconductor.org/packages/3.20/data/annotation/src/contrib/IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz
Resolving bioconductor.org (bioconductor.org)... 3.164.240.108, 3.164.240.121, 3.164.240.104, ...
Connecting to bioconductor.org (bioconductor.org)|3.164.240.108|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://mghp.osn.xsede.org/bir190004-bucket01/archive.bioconductor.org/packages/3.20/data/annotation/src/contrib/IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz [following]
--2025-06-24 14:49:20--  https://mghp.osn.xsede.org/bir190004-bucket01/archive.bioconductor.org/packages/3.20/data/annotation/src/contrib/IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz
Resolving mghp.osn.xsede.org (mghp.osn.xsede.org)... 192.69.103.248, 192.69.103.247, 192.69.103.246
Connecting to mghp.osn.xsede.org (mghp.osn.xsede.org)|192.69.103.248|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 156309587 (149M) [application/gzip]
Saving to: ‘IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz’

IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0. 100%[=================================================================================================================>] 149.07M  33.8MB/s    in 8.1s

2025-06-24 14:49:29 (18.4 MB/s) - ‘IlluminaHumanMethylationEPICv2anno.20a1.hg38_1.0.0.tar.gz’ saved [156309587/156309587]

Now, either it's a bug that bioconductur.org serves a 302 for this file, or it's a bug in the skeleton script.

If you remove -b 3.20 it succeeds, but I want to enable this package for 3.20 so this is not a solution for me.