Add example of different packages per python version

[michaeljs1990]

This is me trying to add an example of something that I was seeing not work as I expected internally and making sure it wasn't something specific to our repo. In our usecase we actually have a package interal that only works up to a specific version of python.

However I keep getting this error when using py_binary or py_test.

ERROR: /private/var/tmp/_bazel_schuettm/e053d0eaf0a843e9f76bdb61766af4b7/external/rules_python++pip+pypi_3_10_only/flask/BUILD.bazel:5:12: configurable attribute "actual" in @@rules_python++pip+pypi_3_10_only//flask:pkg doesn't match this configuration: No matching wheel for current configuration's Python version.

The current build configuration's Python version doesn't match any of the Python
wheels available for this distribution. This distribution supports the following Python
configuration settings:
    //_config:is_python_3.11

To determine the current configuration's Python version, run:
    `bazel config <config id>` (shown further below)

and look for one of:
    @@rules_python+//python/config_settings:python_version
    @@rules_python+//python/config_settings:pip_whl
    @@rules_python+//python/config_settings:pip_whl_glibc_version
    @@rules_python+//python/config_settings:pip_whl_muslc_version
    @@rules_python+//python/config_settings:pip_whl_osx_arch
    @@rules_python+//python/config_settings:pip_whl_osx_version
    @@rules_python+//python/config_settings:py_freethreaded
    @@rules_python+//python/config_settings:py_linux_libc

If the value is missing, then the default value is being used, see documentation:
https://rules-python.readthedocs.io/en/latest/api/rules_python/python/config_settings

This instance of @@rules_python++pip+pypi_3_10_only//flask:pkg has configuration identifier e62ee83. To inspect its configuration, run: bazel config e62ee83.

For more help, see https://bazel.build/docs/configurable-attributes#faq-select-choose-condition.

Use --verbose_failures to see the command lines of failed build steps.
ERROR: Analysis of target '//tests:my_lib_3_10_only_test' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.197s, Critical Path: 0.02s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully
//tests:test_versions                                                 NO STATUS

Executed 0 out of 1 test: 1 was skipped.

Following the directions hasn't helped me but maybe it makes more sense to someone else.

PR Instructions/requirements

• Title uses type: description format. See CONTRIBUTING.md for types.
  • Common types are: build, docs, feat, fix, refactor, revert, test
  • Update CHANGELOG.md as applicable
• Breaking changes include "!" after the type and a "BREAKING CHANGES:"
  section at the bottom.
  See CONTRIBUTING.md for our breaking changes process.
• Body text describes:
  • Why this change is being made, briefly.
  • Before and after behavior, as applicable
  • References issue number, as applicable
• Update docs and tests, as applicable
• Delete these instructions prior to sending the PR

#2530

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[aignas]

Just to write here what I wrote in a related thread on Slack. I think what is happening here is:

• The default python toolchain version is 3.11 and if you build any target in the repo that is not using transitions, //python/version:python_version will just use that.
• This means that the flask here will be evaluated and will fail because there is no set of dependencies for 3.11 in the deps list.

There are ways to work around it:

• Set the target_compatible_with for this target, see the suggestion below.
• Set tags = ["manual"] and create a separate target that is transitioning the my_310_only_lib to something a build configuration where python_version = 3.10.
• Add a pip.parse entry for the default python toolchain version.
• Make 3.10 the default toolchain version.

The TCW snippet:

Suggested change

	],
	],
	target_compatible_with = select({
	"@rules_python//python/config_settings:is_python_3.10": [],
	"//conditions:default": ["@platforms//:incompatible"],
	})

Not sure exactly how rules_python can help here best. The error allows to ensure that the packages are getting the versions that they expect, but I see how this might be adding unnecessary work on the consumer side.

Maybe there should also be a transition py_library implementation?

[michaeljs1990]

I am not sure if it is the right choice but I did look for a version specific implementation of py_library to import when I first hit this and didn't see one. I also think just adding it in here at least for me as an adequate solution and is easy to point people to in the future. After I saw py_library didn't exist for the given version I looked through the examples here to see if someone had already shown how this could be done.

[aignas]

The more I think about it the more I am convinced that users have one of 3 options available to them and this example should use that:

• Set the target_compatible_with for this target, see the suggestion below.
• Set tags = ["manual"] and create a separate target that is transitioning the my_310_only_lib to something a build configuration where python_version = 3.10.
• Make the toolchain version that is used in py_library instances like this the default, or add a pip.parse entry for the default python toolchain version.

I don't think there is anything wrong in this case with rules_python, so I am interested in hearing your feedback as to what makes the above options not feasible in your case.

[michaeljs1990]

Sorry for the miscommunication I think the use of target_compatible_with is a perfectly reasonable solution and I already made this change in our internal repo and things are working as expected.

The only thing I am running into now is figuring out how to make this work with the WORKSPACE based build in this repo. We just use Bazel modules internally so I didn't have to mess with that.

[JeroenSchmidt]

[coming from slack thread]
Hi @michaeljs1990
Thanks for making this example MR! Does it already work when building a py_library with bzlmod?

I checkout out your branch and ran bazel build //libs/my_310_only_lib:my_310_only_lib from the example root of multi_python_versions but I'm getting the same error as I got earlier when I tried target_compatible_with

//libs/my_310_only_lib:my_310_only_lib (968159)   <-- target platform (@@platforms//host:host) didn't satisfy constraint @@platforms//:incompatible

For good measure I also ran bazel build //libs/my_310_only_lib:my_310_only_lib --noenable_workspace and got the same erroras above.

Having said that, the following works: bazel build //...

[michaeljs1990]

[coming from slack thread] Hi @michaeljs1990 Thanks for making this example MR! Does it already work when building a py_library with bzlmod?

I checkout out your branch and ran bazel build //libs/my_310_only_lib:my_310_only_lib from the example root of multi_python_versions but I'm getting the same error as I got earlier when I tried target_compatible_with

//libs/my_310_only_lib:my_310_only_lib (968159)   <-- target platform (@@platforms//host:host) didn't satisfy constraint @@platforms//:incompatible

For good measure I also ran bazel build //libs/my_310_only_lib:my_310_only_lib --noenable_workspace and got the same erroras above.

Having said that, the following works: bazel build //...

Ah I did not try to run the target directly as my tests ran just fine but I think the failure here makes sense because it's going to try to use the default python that is configured when you run the build. This makes sense but means you need to run the tests to make sure your build is working.

bazel build --@rules_python//python/config_settings:python_version=3.10 //libs/my_310_only_lib:my_310_only_lib

for example will pass since you are giving it the correct python version. These are the fixes @aignas is proposing above I believe however I think in that case running the library build directly will still cause a failure but you will be able to verify the sanity of your build with just Bazel build which is ideal.