Include and Exclude statements are ignored

[tamersay]

source: conda-forge
destination: ./my-channel

include:
  - jupyter-ai
exclude:
  - name-glob: jupyter*

This is from the examples. I ran it and saw that conda-mirror basically tries to mirror all the packages in the conda-forge.

[00:00:02] #--------------------------------------- 66/293306 Mirroring

I believe this is not expected.

I tried this example, too

source: conda-forge
destination: /usr/share/nginx/html/miniconda/conda-forge-test

subdirs:
  - linux-64
  - noarch

exclude:
  - name-glob: "*"
    matchspec: "[license=GPL-3.0-or-later]"

include:
  - numpy

This is acting the same way as above.
🪞 Mirroring conda-forge to /usr/share/nginx/html/miniconda/conda-forge-test...
[00:00:07] #--------------------------------------- 152/289969 Mirroring

PS: I use the prebuilt conda-mirror-x86_64-unknown-linux-musl

I am trying to mirror all the packages in conda-forge that are built for py37 or py39. Is there a way to achieve it?

exclude:
  - name-glob: "*"
    matchspec: "[build=<py2*]"

include:
   - mylistofpackages

This kinda solution is what I need but the above trials already yield unexpected results. Would you be able to check?

[pavelzw]

I ran it and saw that conda-mirror basically tries to mirror all the packages in the conda-forge

this is expected. the readme says

Exclude all packages defined in exclude, override this behavior by specifying overrides in include

this means: download all packages except the ones in exclude but still download the ones that are defined in include.

so this would in the example from the readme download (all packages except jupyter-*) plus jupyter-ai.

I tried this example, too

this would exclude all GPL-3 packages. since numpy is not GPL-3 anyway, the include doesn't matter in that case.

---

i know that one might interpret it differently when exclude + include are provided but that's exactly why i wrote the sentence explaining it. i'm open for adding support for the other way around as well if you propose a sensible way of configuring it like this.

I am trying to mirror all the packages in conda-forge that are built for py37 or py39. Is there a way to achieve it?

source: conda-forge
destination: s3://rattler-testing/conda-forge-py37py39

include:
  - '*[build=py37*]'
  - '*[build=py39*]'

s3-config:
  destination:
    endpoint-url: https://s3.eu-central-1.amazonaws.com
    force-path-style: false
    region: eu-central-1

that should do the trick of including all packages with build string py37* like

$ pixi search 'numpy[build=py37*]'
Using channels: conda-forge

numpy-1.21.6-py37h976b520_0
---------------------------

Name                numpy
Version             1.21.6
Build               py37h976b520_0
Size                6400218
License             BSD-3-Clause
Subdir              linux-64
File Name           numpy-1.21.6-py37h976b520_0.tar.bz2
URL                 https://conda.anaconda.org/conda-forge/linux-64/numpy-1.21.6-py37h976b520_0.tar.bz2
MD5                 b0701d8dc4849b6e06caf1c84135c13c
SHA256              89ff0557d8e55bd4944c0c283a91c18a2be62305b4b26a0a8a6e0938e550f7ee

Dependencies:
 - libblas >=3.8.0,<4.0a0
 - libcblas >=3.8.0,<4.0a0
 - libgcc-ng >=10.3.0
 - liblapack >=3.8.0,<4.0a0
 - libstdcxx-ng >=10.3.0
 - python >=3.7,<3.8.0a0
 - python_abi 3.7.* *_cp37m

Run exports:
  weak:
   - numpy >=1.21.6,<2.0a0

this should include most packages that are compiled against python 3.7
this doesn't include all noarch python packages though, that are compatible with python 3.7
it also doesn't include abi3 packages (although they are a very recent addition to conda-forge and probably not relevant for such old python versions as py37) like py-rattler

in that example, py-rattler-0.16.0-py310h6334b52_1.conda contains py310 at the beginning of its build string but is still compatible with all newer python versions of 3.10 as well

matchspec: "[build=<py2*]" is definitely wrong as the <py2* part needs to be a glob.
i would suggest reading into conda/ceps#82 if you want to know more about matchspecs. while not officially accepted yet, this still is the best documentation for matchspec you will get today.