Merge branch 'main' into enumerate-duplicate-model-names

Author: dbanty

.github/workflows/checks.yml

@@ -17,7 +17,7 @@ jobs:
     steps:
       - uses: actions/[email protected]
       - name: Set up Python
-        uses: actions/setup-python@v5.4.0
+        uses: actions/setup-python@v5.5.0
         with:
           python-version: ${{ matrix.python }}
 
@@ -60,7 +60,7 @@ jobs:
         if: matrix.os == 'ubuntu-latest'
 
       - name: Store coverage report
-        uses: actions/[email protected].1
+        uses: actions/[email protected].2
         if: matrix.os == 'ubuntu-latest'
         with:
           name: coverage-${{ matrix.python }}
@@ -76,7 +76,7 @@ jobs:
     steps:
       - uses: actions/[email protected]
       - name: Set up Python
-        uses: actions/setup-python@v5.4.0
+        uses: actions/setup-python@v5.5.0
         with:
           python-version: "3.9"
 
@@ -117,7 +117,7 @@ jobs:
         with:
           python-version: "3.12"
       - name: Download coverage reports
-        uses: actions/download-artifact@v4.1.9
+        uses: actions/download-artifact@v4.2.1
         with:
           merge-multiple: true
 
@@ -142,7 +142,7 @@ jobs:
           .venv/bin/python -m coverage report --fail-under=100          
 
       - name: Upload HTML report if check failed.
-        uses: actions/[email protected].1
+        uses: actions/[email protected].2
         with:
           name: html-report
           path: htmlcov
@@ -164,7 +164,7 @@ jobs:
     steps:
       - uses: actions/[email protected]
       - name: Set up Python
-        uses: actions/setup-python@v5.4.0
+        uses: actions/setup-python@v5.5.0
         with:
           python-version: "3.9"
       - name: Get Python Version

CHANGELOG.md

@@ -13,6 +13,77 @@ Programmatic usage of this project (e.g., importing it as a Python module) and t
 
 The 0.x prefix used in versions for this project is to indicate that breaking changes are expected frequently (several times a year). Breaking changes will increment the minor number, all other changes will increment the patch number. You can track the progress toward 1.0 [here](https://github.com/openapi-generators/openapi-python-client/projects/2).
 
+## 0.24.3 (2025-03-31)
+
+### Features
+
+#### Adding support for named integer enums
+
+##1214 by @barrybarrette
+
+Adding support for named integer enums via an optional extension, `x-enum-varnames`. 
+
+This extension is added to the schema inline with the `enum` definition:
+```
+"MyEnum": {
+    "enum": [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        99
+    ],
+    "type": "integer",
+    "format": "int32",
+    "x-enum-varnames": [
+        "Deinstalled",
+        "Installed",
+        "Upcoming_Site",
+        "Lab_Site",
+        "Pending_Deinstall",
+        "Suspended",
+        "Install_In_Progress",
+        "Unknown"
+    ]
+}
+```
+
+The result:
+![image](https://github.com/user-attachments/assets/780880b3-2f1f-49be-823b-f9abb713a3e1)
+
+## 0.24.2 (2025-03-22)
+
+### Fixes
+
+#### Make lists of models and enums work correctly in custom templates
+
+Lists of model and enum classes should be available to custom templates via the Jinja
+variables `openapi.models` and `openapi.enums`, but these were being passed in a way that made
+them always appear empty. This has been fixed so a custom template can now iterate over them.
+
+Closes #1188.
+
+## 0.24.1 (2025-03-15)
+
+### Features
+
+- allow Ruff to 0.10 (#1220)
+- allow Ruff 0.11 (#1222)
+- Allow any `Mapping` in generated `from_dict` functions (#1211)
+
+### Fixes
+
+#### Always parse `$ref` as a reference
+
+If additional attributes were included with a `$ref` (for example `title` or `description`), the property could be 
+interpreted as a new type instead of a reference, usually resulting in `Any` in the generated code.
+Now, any sibling properties to `$ref` will properly be ignored, as per the OpenAPI specification.
+
+Thanks @nkrishnaswami!
+
 ## 0.24.0 (2025-03-03)
 
 ### Breaking Changes

CONTRIBUTING.md

@@ -50,26 +50,40 @@ All changes must be tested, I recommend writing the test first, then writing the
 
 If you think that some of the added code is not testable (or testing it would add little value), mention that in your PR and we can discuss it.
 
-1. If you're adding support for a new OpenAPI feature or covering a new edge case, add an [end-to-end test](#end-to-end-tests)
-2. If you're modifying the way an existing feature works, make sure an existing test generates the _old_ code in `end_to_end_tests/golden-record`. You'll use this to check for the new code once your changes are complete.
-3. If you're improving an error or adding a new error, add a [unit test](#unit-tests)
+1. If you're adding support for a new OpenAPI feature or covering a new edge case, add [functional tests](#functional-tests), and optionally an [end-to-end snapshot test](#end-to-end-snapshot-tests).
+2. If you're modifying the way an existing feature works, make sure functional tests cover this case. Existing end-to-end snapshot tests might also be affected if you have changed what generated model/endpoint code looks like.
+3. If you're improving error handling or adding a new error, add [functional tests](#functional-tests).
+4. For tests of low-level pieces of code that are fairly self-contained, and not tightly coupled to other internal implementation details, you can use regular [unit tests](#unit-tests).
 
-#### End-to-end tests
+#### End-to-end snapshot tests
 
-This project aims to have all "happy paths" (types of code which _can_ be generated) covered by end to end tests (snapshot tests). In order to check code changes against the previous set of snapshots (called a "golden record" here), you can run `pdm e2e`. To regenerate the snapshots, run `pdm regen`.
+This project aims to have all "happy paths" (types of code which _can_ be generated) covered by end-to-end tests. There are two types of these: snapshot tests, and functional tests.
 
-There are 4 types of snapshots generated right now, you may have to update only some or all of these depending on the changes you're making. Within the `end_to_end_tets` directory:
+Snapshot tests verify that the generated code is identical to a previously-committed set of snapshots (called a "golden record" here). They are basically regression tests to catch any unintended changes in the generator output.
+
+In order to check code changes against the previous set of snapshots (called a "golden record" here), you can run `pdm e2e`. To regenerate the snapshots, run `pdm regen`.
+
+There are 4 types of snapshots generated right now, you may have to update only some or all of these depending on the changes you're making. Within the `end_to_end_tests` directory:
 
 1. `baseline_openapi_3.0.json` creates `golden-record` for testing OpenAPI 3.0 features
 2. `baseline_openapi_3.1.yaml` is checked against `golden-record` for testing OpenAPI 3.1 features (and ensuring consistency with 3.0)
 3. `test_custom_templates` are used with `baseline_openapi_3.0.json` to generate `custom-templates-golden-record` for testing custom templates
 4. `3.1_specific.openapi.yaml` is used to generate `test-3-1-golden-record` and test 3.1-specific features (things which do not have a 3.0 equivalent)
 
+#### Functional tests
+
+These are black-box tests that verify the runtime behavior of generated code, as well as the generator's validation behavior. They are also end-to-end tests, since they run the generator as a shell command.
+
+This can sometimes identify issues with error handling, validation logic, module imports, etc., that might be harder to diagnose via the snapshot tests, especially during development of a new feature. For instance, they can verify that JSON data is correctly decoded into model class attributes, or that the generator will emit an appropriate warning or error for an invalid spec.
+
+See [`end_to_end_tests/functional_tests`](./end_to_end_tests/functional_tests).
+
 #### Unit tests
 
-> **NOTE**: Several older-style unit tests using mocks exist in this project. These should be phased out rather than updated, as the tests are brittle and difficult to maintain. Only error cases should be tests with unit tests going forward.
+These include:
 
-In some cases, we need to test things which cannot be generated—like validating that errors are caught and handled correctly. These should be tested via unit tests in the `tests` directory, using the `pytest` framework.
+* Regular unit tests of basic pieces of fairly self-contained low-level functionality, such as helper functions. These are implemented in the `tests` directory, using the `pytest` framework.
+* Older-style unit tests of low-level functions like `property_from_data` that have complex behavior. These are brittle and difficult to maintain, and should not be used going forward. Instead, they should be migrated to functional tests.
 
 ### Creating a Pull Request
 

README.md

@@ -198,6 +198,39 @@ content_type_overrides:
   application/zip: application/octet-stream
 ```
 
+## Supported Extensions
+
+### x-enum-varnames
+
+This extension has been adopted by similar projects such as [OpenAPI Tools](https://github.com/OpenAPITools/openapi-generator/pull/917).
+It is intended to provide user-friendly names for integer Enum members that get generated.
+It is critical that the length of the array matches that of the enum values.
+
+```
+"Colors": {
+   "type": "integer",
+   "format": "int32",
+   "enum": [
+       0,
+       1,
+       2
+   ], 
+  "x-enum-varnames": [
+      "Red",
+      "Green",
+      "Blue"
+   ]
+}
+```
+
+Results in:
+```
+class Color(IntEnum):
+    RED = 0
+    GREEN = 1
+    BLUE = 2
+```
+
 [changelog.md]: CHANGELOG.md
 [poetry]: https://python-poetry.org/
 [PDM]: https://pdm-project.org/latest/

end_to_end_tests/__init__.py

@@ -1 +1,5 @@
 """ Generate a complete client and verify that it is correct """
+import pytest
+
+pytest.register_assert_rewrite("end_to_end_tests.end_to_end_test_helpers")
+pytest.register_assert_rewrite("end_to_end_tests.functional_tests.helpers")

end_to_end_tests/baseline_openapi_3.0.json

@@ -394,7 +394,16 @@
           "content": {
             "multipart/form-data": {
               "schema": {
-                "$ref": "#/components/schemas/Body_upload_file_tests_upload_post"
+                "$ref": "#/components/schemas/Body_upload_file_tests_upload_post",
+                "title": "Body_upload_file_tests_upload_post",
+                "required": [
+                  "some_file",
+                  "some_object",
+                  "some_nullable_object",
+                  "some_required_number"
+                ],
+                "properties": {
+                }
               }
             }
           },

end_to_end_tests/custom-templates-golden-record/my_test_api_client/models/__init__.py

@@ -0,0 +1,16 @@
+# Testing that we can access model-related information via Jinja variables.
+
+# To avoid having to update this file in the golden record every time the test specs are changed,
+# we won't include all the classes in this output - we'll just look for one of them.
+
+# Using "alls"
+#   AModel
+
+# Using "imports"
+#   from .a_model import AModel
+
+# Using "openapi.models"
+#   AModel (a_model)
+
+# Using "openapi.enums"
+#   AnEnum (an_enum)

end_to_end_tests/docstrings-on-attributes-golden-record/my_test_api_client/models/model_with_description.py

@@ -1,3 +1,4 @@
+from collections.abc import Mapping
 from typing import Any, TypeVar, Union
 
 from attrs import define as _attrs_define
@@ -43,8 +44,8 @@ def to_dict(self) -> dict[str, Any]:
         return field_dict
 
     @classmethod
-    def from_dict(cls: type[T], src_dict: dict[str, Any]) -> T:
-        d = src_dict.copy()
+    def from_dict(cls: type[T], src_dict: Mapping[str, Any]) -> T:
+        d = dict(src_dict)
         prop_with_no_desc = d.pop("propWithNoDesc", UNSET)
 
         prop_with_desc = d.pop("propWithDesc", UNSET)

end_to_end_tests/docstrings-on-attributes-golden-record/my_test_api_client/models/model_with_no_description.py

@@ -1,3 +1,4 @@
+from collections.abc import Mapping
 from typing import Any, TypeVar, Union
 
 from attrs import define as _attrs_define
@@ -31,8 +32,8 @@ def to_dict(self) -> dict[str, Any]:
         return field_dict
 
     @classmethod
-    def from_dict(cls: type[T], src_dict: dict[str, Any]) -> T:
-        d = src_dict.copy()
+    def from_dict(cls: type[T], src_dict: Mapping[str, Any]) -> T:
+        d = dict(src_dict)
         prop_with_no_desc = d.pop("propWithNoDesc", UNSET)
 
         prop_with_desc = d.pop("propWithDesc", UNSET)

end_to_end_tests/functional_tests/README.md

@@ -0,0 +1,75 @@
+## The `functional_tests` module
+
+These are end-to-end tests which run the client generator against many small API documents that are specific to various test cases.
+
+Rather than testing low-level implementation details (like the unit tests in `tests`), or making assertions about the exact content of the generated code (like the "golden record"-based end-to-end tests), these treat both the generator and the generated code as black boxes and make assertions about their behavior.
+
+The tests are in two submodules:
+
+# `generated_code_execution`
+
+These tests use valid API specs, and after running the generator, they _import and execute_ pieces of the generated code to verify that it actually works at runtime.
+
+Each test class follows this pattern:
+
+- Use the decorator `@with_generated_client_fixture`, providing an inline API spec (JSON or YAML) that contains whatever schemas/paths/etc. are relevant to this test class. 
+  - The spec can omit the `openapi:`, `info:`, and `paths:`, blocks, unless those are relevant to the test.
+  - The decorator creates a temporary file for the inline spec and a temporary directory for the generated code, and runs the client generator.
+  - It creates a `GeneratedClientContext` object (defined in `end_to_end_test_helpers.py`) to keep track of things like the location of the generated code and the output of the generator command.
+  - This object is injected into the test class as a fixture called `generated_client`, although most tests will not need to reference the fixture directly.
+  - `sys.path` is temporarily changed, for the scope of this test class, to allow imports from the generated code.
+- Use the decorator `@with_generated_code_imports` or `@with_generated_code_import` to make classes or functions from the generated code available to the tests.
+  - `@with_generated_code_imports(".models.MyModel1", ".models.MyModel2)` would execute `from [package name].models import MyModel1, MyModel2` and inject the imported classes into the test class as fixtures called `MyModel1` and `MyModel2`.
+  - `@with_generated_code_import(".api.my_operation.sync", alias="endpoint_method")` would execute `from [package name].api.my_operation import sync`, but the fixture would be named `endpoint_method`.
+  - After the test class finishes, these imports are discarded.
+
+Example:
+
+```python
+@with_generated_client_fixture(
+"""
+components:
+  schemas:
+    MyModel:
+      type: object
+      properties:
+        stringProp: {"type": "string"}
+""")
+@with_generated_code_import(".models.MyModel")
+class TestSimpleJsonObject:
+    def test_encoding(self, MyModel):
+        instance = MyModel(string_prop="abc")
+        assert instance.to_dict() == {"stringProp": "abc"}
+```
+
+# `generator_failure_cases`
+
+These run the generator with an invalid API spec and make assertions about the warning/error output. Some of these invalid conditions are expected to only produce warnings about the affected schemas, while others are expected to produce fatal errors that terminate the generator.
+
+For warning conditions, each test class uses `@with_generated_client_fixture` as above, then uses `assert_bad_schema` to parse the output and check for a specific warning message for a specific schema name.
+
+```python
+@with_generated_client_fixture(
+"""
+components:
+  schemas:
+    MyModel:
+      # some kind of invalid schema
+""")
+class TestBadSchema:
+    def test_encoding(self, generated_client):
+        assert_bad_schema(generated_client, "MyModel", "some expected warning text")
+```
+
+Or, for fatal error conditions:
+
+- Call `inline_spec_should_fail`, providing an inline API spec (JSON or YAML).
+
+```python
+class TestBadSpec:
+    def test_some_spec_error(self):
+        result = inline_spec_should_fail("""
+# some kind of invalid spec
+""")
+        assert "some expected error text" in result.output
+```