Currently we have upper bounds for packaging and scikit-build. Removing the upper bound on packaging leads to python not finding scikit-build and a build failure (even if we relax the constraint on scikit-build).
We should investigate which combination of build packages works and try to upgrade to the latest ones.