Merge pull request #116 from DigitalHarborFoundation/docs/fix-peewee-attributes-issue-86

Fix peewee model attributes missing from generated docs (#86)

Author: levon003

.github/workflows/github-pages.yml

@@ -36,6 +36,7 @@ jobs:
         uses: astral-sh/setup-uv@v5
         with:
           version: "latest"
+          enable-cache: true
 
       - name: "Set up Python"
         uses: actions/setup-python@v5

.github/workflows/validate.yaml

@@ -15,6 +15,7 @@ jobs:
         uses: astral-sh/setup-uv@v5
         with:
           version: "latest"
+          enable-cache: true
 
       - name: "Set up Python"
         uses: actions/setup-python@v5
@@ -29,4 +30,34 @@ jobs:
         run: |
           uv run python -m unittest discover -s tests.unit
         env:
-          CURRENT_BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
+          CURRENT_BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
+
+  docs:
+    name: Build docs
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          version: "latest"
+          enable-cache: true
+
+      - name: "Set up Python"
+        uses: actions/setup-python@v5
+        with:
+          python-version-file: ".python-version"
+
+      - name: Install docs dependencies
+        run: |
+          uv sync --group docs
+
+      # Builds the same way github-pages.yml deploys, so doc-build breakage
+      # (and, with -W, doc warnings) is caught on PRs instead of only on the
+      # deploy-to-main step. This job does not deploy.
+      - name: Build docs
+        run: |
+          make html

DEVELOPMENT.md

@@ -149,21 +149,69 @@ uv run python -m flexeval --eval_name {eval_suite_name}
 
 ## Documentation
 
-We use Sphinx to generate the documentation website.
+We use [Sphinx](https://www.sphinx-doc.org/) to generate the [documentation
+website](https://digitalharborfoundation.github.io/FlexEval). The source lives in
+`docs/`, the configuration is `docs/conf.py`, and the relevant build targets are in
+the `Makefile`. The site is rebuilt and deployed to GitHub Pages on every push to
+`main` by the [`github-pages`](.github/workflows/github-pages.yml) workflow.
 
-All of the relevant directives are in the `Makefile`.
+The API reference is generated automatically by `autodoc`/`autosummary` from the
+source code (see the `:recursive:` `autosummary` directive in `docs/api.rst`), so
+new modules and classes show up without any manual wiring.
 
-Develop the documentation website locally:
+### Building the docs locally
+
+First make sure the docs dependencies are installed (they live in the `docs`
+dependency group; `uv sync --all-groups` installs them too):
 
 ```bash
-make docclean docautobuild
+uv sync --group docs
+```
+
+Build the site once:
+
+```bash
+make html
+```
+
+The output is written to `build/html/`; open `build/html/index.html` in a browser to
+view it.
+
+For an interactive workflow, use `sphinx-autobuild`, which rebuilds on save and serves
+the site at <http://127.0.0.1:8000>:
+
+```bash
+make docautobuild
 ```
 
-You can also just build the site directly:
+### Debugging the build
+
+The generated API stubs (`docs/generated/`) and the rendered output (`build/`) are
+**cached** between builds. `autosummary` will not regenerate a stub that already
+exists, so when you change `docs/conf.py` (especially `autodoc`/`autosummary`
+options) or restructure modules, an incremental build can show stale results. Force a
+clean rebuild:
+
 ```bash
+make docclean   # removes build/ and docs/generated/
 make html
+# or chain them, e.g. for autobuild:
+make docclean docautobuild
 ```
 
+Sphinx prints `WARNING:` lines during the build and ends with a summary
+(e.g. `build succeeded, N warnings.`). Warnings are worth scanning — they flag broken
+cross-references, malformed docstrings, and members that failed to render.
+
+A note on the API reference: FlexEval's database models (`src/flexeval/classes/`) are
+[peewee](https://docs.peewee-orm.com/) models, and peewee's metaclass rewrites
+field definitions into descriptors and adds generated members (a per-model
+`DoesNotExist` exception and a `<fk>_id` alias for every foreign key). The
+`skip_peewee_internals` hook in `docs/conf.py` hides that generated noise while
+keeping the real fields, and `inherited-members: False` keeps inherited peewee/pydantic
+machinery out of the reference. If model attributes stop appearing on a generated page,
+that hook (and the `autodoc_default_options`) is the place to look.
+
 ## Releasing a new version
 
 The package version lives in `src/flexeval/__about__.py` (`pyproject.toml` reads it

docs/conf.py

@@ -8,6 +8,8 @@
 from pathlib import Path
 from packaging.version import parse as parse_version
 
+import peewee as pw
+
 import flexeval
 
 sys.path.append(os.path.abspath("."))
@@ -168,6 +170,12 @@ def linkcode_resolve(domain, info):
 nb_merge_streams = True
 
 autosummary_generate = True
+# Don't let numpydoc inject its own per-class "Methods"/"Attributes" summary
+# tables. They duplicate the member documentation autodoc already renders below,
+# and for our peewee models they list every inherited peewee.Model method
+# (save, select, bulk_create, ...) as noise. Disabling this also avoids the
+# "stub file not found" warnings those tables' :toctree: would otherwise emit.
+numpydoc_show_class_members = False
 autodoc_typehints = "signature"
 autodoc_default_options = {
     "members": True,
@@ -178,18 +186,29 @@ def linkcode_resolve(domain, info):
 }
 
 
-def skip_inherited_members(app, what, name, obj, skip, options):
-    # Skip members if they are inherited (not defined on the class itself)
-    if what == "class":
-        # The object is the class being documented
-        cls = obj
-        if hasattr(cls, "__dict__"):
-            # If the member name is NOT in the class dict, it's inherited
-            if name not in cls.__dict__:
-                return True  # skip inherited member
-    return skip  # otherwise use default behavior
+def skip_peewee_internals(app, what, name, obj, skip, options):
+    """Hide peewee-generated noise from the API docs.
+
+    peewee's model metaclass adds two kinds of members to every model class
+    that aren't useful in the generated reference:
+
+    - a per-model ``DoesNotExist`` exception (e.g. ``MetricDoesNotExist``), and
+    - a ``<fk>_id`` alias for every foreign key (e.g. ``dataset_id`` alongside
+      ``dataset``). The alias shares the same ``Field`` object as the FK, whose
+      ``.name`` is the FK field name, so we can detect it by name mismatch.
+
+    Genuine fields and methods are left untouched. (Inherited members are
+    excluded separately via ``inherited-members: False`` below — note that
+    peewee's per-model ``DoesNotExist`` and ``_id`` accessors are defined on the
+    model class itself, not inherited, which is why they need explicit skipping.)
+    """
+    if name == "DoesNotExist":
+        return True
+    if isinstance(obj, pw.ForeignKeyField) and name != obj.name:
+        return True
+    return skip
 
 
 def setup(app):
-    app.connect("autodoc-skip-member", skip_inherited_members)
+    app.connect("autodoc-skip-member", skip_peewee_internals)
     app.connect("builder-inited", vignettes.generate_custom_stubs)