Add testing stage for trying against lowest pinned versions. (#515)

* Add testing stage for trying against lowest pinned versions.

* Update copier.yml

Co-authored-by: Konstantin Malanchev <[email protected]>

---------

Co-authored-by: Konstantin Malanchev <[email protected]>

Author: delucchi-cmu

copier.yml

@@ -64,7 +64,7 @@ project_license:
 
 python_versions:
     help: What versions of python are you targeting? We will add automated testing for these versions.
-    default: ["3.9", "3.10", "3.11"]
+    default: ["3.9", "3.10", "3.11", "3.12", "3.13"]
     type: str
     multiselect: true
     choices:
@@ -145,6 +145,15 @@ include_benchmarks:
         no: false
     when: "{{ custom_install }}"
 
+test_lowest_version:
+    help: Run pull request tests with the lowest versions of python and dependencies? Recommended if you are developing a library
+    default: direct
+    choices:
+        Do not test with lowest versions of dependencies: none
+        Test with lowest versions of direct dependencies only (those listed in pyproject.toml): direct
+        Test with lowest versions of all dependencies: all
+    when: "{{ custom_install }}"
+
 _message_after_copy: |
     Your project "{{ project_name }}" has been created successfully!
 

docs/source/template_options.rst

@@ -125,11 +125,14 @@ The license type you want to use for this project.
 8. Versions of Python
 ---------------------
 
-   +------------+-----------------------------------------------------------------+
-   | Question   | What versions of Python will your project support?              |
-   +------------+-----------------------------------------------------------------+
-   | Options    | 3.7 (end-of-life), 3.8, **✱ 3.9**, **✱ 3.10**, **✱ 3.11**, 3.12 |
-   +------------+-----------------------------------------------------------------+
+   +------------+----------------------------------------------------+
+   | Question   | What versions of Python will your project support? |
+   +------------+----------------------------------------------------+
+   | Options    | 3.8 (end-of-life), **✱ 3.9**,                      |
+   |            | **✱ 3.9**, **✱ 3.10**,                             |
+   |            | **✱ 3.11**, **✱ 3.12**,                            |
+   |            | 3.13                                               |
+   +------------+----------------------------------------------------+
 
 Select all versions of python that you are targeting for execution.
 
@@ -271,3 +274,27 @@ It will also create a sample benchmarking suite under ``benchmarks/``::
     ├─ ...
 
 Read more at :doc:`../practices/ci_benchmarking`.
+
+16. Test against lowest versions
+------------------------------------------------
+
+   +------------+-----------------------------------------------------------+
+   | Question   | Run pull request tests with the lowest versions of python |
+   |            | and dependencies?                                         |
+   +------------+-----------------------------------------------------------+
+   | Options    | | (none) Do not test with lowest versions of dependencies | 
+   |            | | **(direct)** Test with lowest versions of direct        |
+   |            |   dependencies only (those listed in pyproject.toml)      | 
+   |            | | (all) Test with lowest versions of all dependencies     |
+   +------------+-----------------------------------------------------------+
+
+Adds an optional stage to the end of the testing and coverage github CI workflow
+using the oldest version of python you've selected above, and will determine 
+the oldest versions of dependencies indicated in the pyproject file.
+
+A new environment has these dependencies installed, and we try to run the pytest
+unit tests against these versions.
+
+This can be useful if your project is a library, and folks may want to depend on 
+you and your dependencies. Understanding the true range of versions you support
+in your application can make dependency resolution much easier for your users!

python-project-template/.github/workflows/testing-and-coverage.yml.jinja

@@ -1,3 +1,4 @@
+{%- import 'python-versions.jinja' as py %}
 # This workflow will install Python dependencies, run tests and report code coverage with a variety of Python versions
 # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
 
@@ -36,3 +37,29 @@ jobs:
       uses: codecov/codecov-action@v5
       with:
         token: {% raw %}${{ secrets.CODECOV_TOKEN }}{% endraw %}
+{%- if test_lowest_version != 'none' %}
+  test-lowest-versions:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python {{ py.min(python_versions)  }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: '{{ py.min(python_versions)  }}'
+    - name: Install dependencies
+      run: |
+        sudo apt-get update
+        python -m pip install --upgrade uv
+        uv venv venv
+        source venv/bin/activate
+        {%- if test_lowest_version == 'direct' %}
+        uv pip compile --resolution=lowest-direct -o requirements_lowest.txt pyproject.toml
+        {%- elif test_lowest_version == 'all' %}
+        uv pip compile --resolution=lowest -o requirements_lowest.txt pyproject.toml
+        {%- endif %}
+        uv pip install --constraint=requirements_lowest.txt -e .[dev]
+    - name: Run unit tests with pytest
+      run: |
+        source venv/bin/activate
+        python -m pytest
+{%- endif %}

tests/test_package_creation.py

@@ -288,6 +288,25 @@ def test_doc_combinations_no_docs(copie, doc_answers):
     assert not (result.project_dir / "docs").is_dir()
 
 
+@pytest.mark.parametrize("test_lowest_version", ["none", "direct", "all"])
+def test_test_lowest_version(copie, test_lowest_version):
+    """Confirm we can generate a "testing_and_coverage.yaml" file, with all
+    test_lowest_version mechanisms selected."""
+
+    # provide a dictionary of the non-default answers to use
+    extra_answers = {
+        "test_lowest_version": test_lowest_version,
+    }
+
+    # run copier to hydrate a temporary project
+    result = copie.copy(extra_answers=extra_answers)
+
+    assert successfully_created_project(result)
+    assert directory_structure_is_correct(result)
+    assert black_runs_successfully(result)
+    assert contains_required_files(result)
+
+
 def test_github_workflows_schema(copie):
     """Confirm the current GitHub workflows have valid schemas."""
     extra_answers = {