When installing or updating packages from the conda-forge channel, conda refuses to install the GPU version regardless of what's specified.
Add CONDA_OVERRIDE_CUDA=11.2 before the conda command. For example,
CONDA_OVERRIDE_CUDA=11.2 conda install jax -c conda-forge
This problem occurs because conda makes a virtual package for the CUDA version it detects.