Skip to content

feat(workflows): add python-qlty-gate reusable workflow#188

Merged
williaby merged 3 commits into
mainfrom
feat/qlty-gate-reusable
Jun 3, 2026
Merged

feat(workflows): add python-qlty-gate reusable workflow#188
williaby merged 3 commits into
mainfrom
feat/qlty-gate-reusable

Conversation

@williaby
Copy link
Copy Markdown
Collaborator

@williaby williaby commented Jun 3, 2026

Summary

Adds a reusable workflow that runs qlty check as a blocking CI gate.

  • Diff mode (default): analyzes only changed files relative to an upstream ref; used for PR gates so only newly introduced issues block merges
  • Full scan mode (check-all: true): runs qlty check --all; used for scheduled health scans to track accumulated debt on the default branch

The job inside the reusable workflow is named Qlty Gate. When called with caller job id qlty-gate (required by convention), the resulting CheckRun name is qlty-gate / Qlty Gate. This is the name to add to required_status_checks in the org baseline ruleset.

Inputs

Input Default Purpose
fail-level medium Minimum severity that triggers non-zero exit
check-all false Full scan vs diff mode
no-fail false Informational mode (always exits 0)
upstream '' Base ref for diff comparison
timeout-minutes 15 Job timeout

Context

Part of a two-PR set to wire up qlty enforcement:

  1. This PR: add the reusable workflow
  2. reference-library PR: call this workflow from qlty.yml with fail-level medium (PR gate) and scheduled full scan

The companion PR references commit 040026ab682aa4b9ef491750d62cdd1592cdb659 from this branch.

Test plan

  • python-qlty-gate.yml validates with actionlint
  • Companion reference-library PR shows qlty-gate / Qlty Gate CheckRun after this merges
  • Verify qlty-gate / Qlty Gate added to ByronWilliamsCPA-default-branch-baseline required checks

Summary by CodeRabbit

  • Chores
    • Added a new reusable CI workflow for code quality checks. Includes configurable fail severity levels, flexible scanning modes (differential or full codebase analysis), upstream base reference specification, custom timeout values, optional informational reporting, and detailed job summaries.

Copilot AI review requested due to automatic review settings June 3, 2026 02:22
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 3, 2026

Review Change Stack

Warning

Review limit reached

@williaby, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 53 minutes and 22 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2ba99458-86d4-4dd5-8d80-abaec42ce47b

📥 Commits

Reviewing files that changed from the base of the PR and between 040026a and 490c0d8.

📒 Files selected for processing (3)
  • .github/workflows/python-qlty-gate.yml
  • CHANGELOG.md
  • docs/workflows/python-qlty-gate.md
📝 Walkthrough

Walkthrough

This PR introduces a new reusable GitHub Actions workflow that standardizes code quality checking via the Qlty CLI. Callers can invoke the workflow with five configurable inputs to run quality checks in either diff mode (comparing against an upstream branch) or full scan mode, with configurable fail levels and reporting.

Changes

Qlty CI Gate Workflow

Layer / File(s) Summary
Workflow interface and metadata
.github/workflows/python-qlty-gate.yml
Documentation describes the reusable gate behavior and two operating modes (diff vs full scan). The workflow_call interface exposes inputs for fail-level, check-all, no-fail, upstream, and timeout-minutes. Permissions are set to minimal read-only access.
Gate job execution and CLI argument construction
.github/workflows/python-qlty-gate.yml
The gate job checks out the repository with full history, installs the Qlty CLI action, and conditionally builds and executes qlty check arguments based on the caller-provided configuration.
Job summary reporting
.github/workflows/python-qlty-gate.yml
An always() step writes a Markdown table to $GITHUB_STEP_SUMMARY reporting the chosen scan mode (full or diff), fail-level configuration, and whether the run is informational.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A gate of Qlty, shiny and new,
Checks code with precision, through and through,
Diff or full scan, the caller decides,
With summaries posted where status resides! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically summarizes the main change: adding a reusable GitHub Actions workflow for qlty quality gates.
Description check ✅ Passed The PR description covers all critical sections from the template with clear context, inputs documentation, and testing plan, though some template checkboxes remain unchecked.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/qlty-gate-reusable

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new reusable GitHub Actions workflow to serve as an organization-wide CI quality gate by running qlty check in either diff mode (PR gating) or full scan mode (scheduled debt tracking), with a stable CheckRun name for ruleset required checks.

Changes:

  • Introduces .github/workflows/python-qlty-gate.yml as a reusable workflow with inputs controlling severity threshold, diff versus full scan behavior, and informational mode.
  • Implements secure-by-default workflow structure with workflow-level permissions: {}, job-level scoped permissions, job timeout, and runner hardening as the first step.
  • Adds a job summary that records the effective gate settings in the workflow run summary.

Comment on lines +108 to +109
with:
fetch-depth: 0
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
.github/workflows/python-qlty-gate.yml (1)

52-58: ⚡ Quick win

Consider adding validation for fail-level input.

The input accepts any string, but line 55 documents only five valid values: note, fmt, low, medium, high. If a caller passes an invalid value, the qlty check command will fail at runtime. Consider adding a validation step early in the job to fail fast with a clear error message.

✅ Proposed validation step

Add this step after checkout and before running qlty:

      - name: Validate inputs
        run: |
          valid_levels="note fmt low medium high"
          if ! echo "$valid_levels" | grep -qw "${{ inputs.fail-level }}"; then
            echo "::error::Invalid fail-level '${{ inputs.fail-level }}'. Must be one of: $valid_levels"
            exit 1
          fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/python-qlty-gate.yml around lines 52 - 58, Add an early
validation step for the action input named fail-level to ensure it is one of the
allowed values ("note", "fmt", "low", "medium", "high") and fail fast with a
clear error if not; insert a new job step after checkout and before running the
qlty command that checks inputs.fail-level against that whitelist (e.g., compare
the input string to the set of valid tokens and emit an error + exit 1 when it
does not match) so callers receive a clear message rather than a runtime qlty
failure.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/python-qlty-gate.yml:
- Around line 106-109: The Checkout step using
actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd must explicitly
disable credential persistence to avoid exposing GITHUB_TOKEN; in the workflow
(the Checkout action block) add the input persist-credentials: false so the
token is not written into .git/config when running qlty check.
- Around line 114-128: The step can run with neither --all nor --upstream which
leaves qlty check mode ambiguous; add validation before building args to detect
when CHECK_ALL != "true" and UPSTREAM is empty and then either set a sensible
default upstream (e.g., origin/${{ github.event.pull_request.base.ref }} for PR
runs) or abort with a clear error and non‑zero exit; implement this by checking
the env vars CHECK_ALL and UPSTREAM, printing a helpful message to stderr
(including the values) and exiting 1 if neither flag will be provided, or by
populating UPSTREAM with the derived default before appending to args so qlty
check always receives either --all or --upstream.

---

Nitpick comments:
In @.github/workflows/python-qlty-gate.yml:
- Around line 52-58: Add an early validation step for the action input named
fail-level to ensure it is one of the allowed values ("note", "fmt", "low",
"medium", "high") and fail fast with a clear error if not; insert a new job step
after checkout and before running the qlty command that checks inputs.fail-level
against that whitelist (e.g., compare the input string to the set of valid
tokens and emit an error + exit 1 when it does not match) so callers receive a
clear message rather than a runtime qlty failure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 44c4fbd2-dd9d-4eb7-8dbe-25ee302c015c

📥 Commits

Reviewing files that changed from the base of the PR and between 30407bf and 040026a.

📒 Files selected for processing (1)
  • .github/workflows/python-qlty-gate.yml

Comment thread .github/workflows/python-qlty-gate.yml
Comment thread .github/workflows/python-qlty-gate.yml
@williaby
Copy link
Copy Markdown
Collaborator Author

williaby commented Jun 3, 2026

PR Review

CI green, SonarCloud Quality Gate passed, no Critical findings. Branch is BEHIND main (rebase before merge). Every quantitative claim below was verified against the repo; two automated findings were dropped (one hallucinated harden-runner version split, one misapplied branch-naming rule).

Important (should fix)

  1. CHANGELOG not updated. This feat adds a new public workflow with no [Unreleased] entry. Recent feat PRs (feat: curated audit remediations (SLSA caller, pre-commit CI gate, operator-script hardening) #185, feat(python-sbom): add OSV-Scanner SBOM-ingest gate + nightly schedule (#152) #179) both updated CHANGELOG, so this breaks a live, enforced convention.
  2. Checkout missing persist-credentials: false (L106-L109) (Copilot + CodeRabbit + zizmor artipacked). GITHUB_TOKEN persists in .git/config while qlty check scans checked-out code. Valid hardening; note no sibling python-*.yml sets this either, so consider a fleet-wide fix.
  3. Ambiguous default mode (L114-L128). With check-all: false and upstream: '' (both defaults) neither --all nor --upstream is passed; the command degrades to bare qlty check --fail-level medium, contradicting the documented diff-mode contract. Fail fast or derive a PR default upstream (CodeRabbit posted a committable guard).
  4. Missing companion doc docs/workflows/python-qlty-gate.md. The docs/workflows/ directory follows a verified 1:1 pattern; this file's doc returns 404.

Suggested: RAD markers absent on permission scopes / reusable inheritance (project standard mandates them, though the analogous python-qlty-coverage.yml also omits them); fail-level input unvalidated against the allowed enum; job summary omits the upstream ref; caller examples use @<sha> vs the sibling @v1 convention.

Verified clean: the set -e + && one-liner does not abort the step (LHS of && is exempt); no command injection (inputs flow through quoted env vars into a bash array).

🤖 Generated with Claude Code

williaby and others added 3 commits June 2, 2026 23:01
Adds a reusable workflow that runs qlty check as a blocking CI gate.
Two modes: diff (PR gate, changed files only) and full scan (scheduled
health checks, --all). The resulting CheckRun is named
"{caller-job-id} / Qlty Gate" -- callers should use job id "qlty-gate"
so the org ruleset required_status_checks entry matches across repos.

Inputs: fail-level (default: medium), check-all, no-fail, upstream,
timeout-minutes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Address review findings on the new reusable qlty gate:

- Set persist-credentials: false on checkout so GITHUB_TOKEN is not written
  to .git/config while qlty check scans checked-out code (Copilot, CodeRabbit,
  zizmor artipacked).
- Fail fast when neither check-all nor upstream is set, instead of running
  qlty check in an ambiguous default scope (CodeRabbit).
- Validate fail-level against the allowed set (note/fmt/low/medium/high) with
  an explicit error rather than an opaque CLI failure.
- Add the upstream ref row to the job summary so diff-mode runs show their base.
- Add RAD markers on the workflow- and job-level permission scopes and on the
  caller/callee inheritance, per the project RAD standard for workflow YAML.
- Point the caller examples at the documented @v1 consumer tag.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add the docs/workflows/python-qlty-gate.md companion page (operating modes,
caller examples, inputs table, required-status-check naming) to match the
1:1 workflow-to-doc pattern, and record the workflow under CHANGELOG
[Unreleased].

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@williaby williaby force-pushed the feat/qlty-gate-reusable branch from 040026a to 490c0d8 Compare June 3, 2026 06:06
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Jun 3, 2026

@williaby
Copy link
Copy Markdown
Collaborator Author

williaby commented Jun 3, 2026

PR Fix Summary

Rebased onto main (was 2 behind) and pushed remediations for the review findings. New head: 490c0d8.

Workflow hardening (364f0e1)

  • persist-credentials: false added to the Checkout step. Addresses the Copilot and CodeRabbit threads and zizmor artipacked: GITHUB_TOKEN is no longer written to .git/config while qlty check scans the tree.
  • Ambiguous default mode fixed. The run step now fails fast with a clear ::error:: when neither check-all nor upstream is set, so diff mode can no longer run with an unspecified scope (CodeRabbit thread). The header doc and the upstream input description now state that upstream is required in diff mode.
  • fail-level validation. Invalid values now fail with an explicit error listing the allowed set (note/fmt/low/medium/high) instead of an opaque qlty CLI failure.
  • Job summary now includes the upstream ref row so diff-mode runs show which base they compared against.
  • RAD markers added on the workflow- and job-level permission scopes and on the caller/callee inheritance, per the project RAD standard for workflow YAML.
  • Caller examples now reference @v1 (the documented consumer tag) instead of an @<sha> placeholder.

Docs (490c0d8)

  • Added docs/workflows/python-qlty-gate.md companion page (operating modes, caller examples, inputs table, required-status-check naming), matching the 1:1 workflow-to-doc pattern.
  • Recorded the workflow under CHANGELOG.md [Unreleased].

Note on the ::error:: annotations: these intentionally write to stdout, not stderr. GitHub Actions only parses ::error:: workflow commands from stdout, so a >&2 redirect would silently drop the annotation.

actionlint (incl. shellcheck on the run blocks), yamllint, markdownlint, and the no-em-dash guard all pass locally. CI re-run triggered by the push.

Threaded replies to the individual review comments could not be posted from the review sandbox (it blocks gh api write calls); this consolidated comment addresses each thread. The three threads (Copilot persist-credentials, CodeRabbit persist-credentials, CodeRabbit ambiguous-mode) are all resolved by 364f0e1.

🤖 Generated with Claude Code

@williaby williaby merged commit 1561a3e into main Jun 3, 2026
27 checks passed
@williaby williaby deleted the feat/qlty-gate-reusable branch June 3, 2026 12:58
williaby added a commit that referenced this pull request Jun 4, 2026
Adds qlty-gate job (PR diff gate, fail-level medium) and qlty-health
job (weekly full scan, informational). No coverage upload — this is
an infra/config repo.
Refs: #188

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit that referenced this pull request Jun 4, 2026
…kflow

Resolve the SonarCloud gate failure on PR #189 (githubactions:S8234, MAJOR):
replace workflow-level 'permissions: read-all' with deny-by-default
'permissions: {}'. Both jobs already declare job-scoped 'contents: read', the
minimum a diff/health gate needs, matching the python-qlty-gate.yml convention.
Also flagged by Copilot.

Re-pin the reusable-workflow refs from 040026a (pre-squash feature-branch commit
of #188, diverged from main) to 1561a3e, the squash-merge commit on main that
contains python-qlty-gate.yml, so the pin is auditable and Renovate-trackable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit that referenced this pull request Jun 4, 2026
Add an Unreleased entry for the qlty.yml caller added in PR #189, matching the
repo pattern where feat changes update CHANGELOG.md (as sibling PR #188 did).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit to ByronWilliamsCPA/.claude that referenced this pull request Jun 4, 2026
Adds CI-078 (suggested, override-eligible): .github/workflows/qlty.yml must
contain a qlty-gate job calling python-qlty-gate.yml. The qlty.sh GitHub App
StatusContext always reports SUCCESS regardless of blocking issue count and
cannot enforce quality gates; a real GitHub Actions CheckRun from the qlty-gate
job is required for enforcement.

Allocated CI-078 (not CI-073) to avoid an ID collision with the renovate-health
suite's CI-073..077 in PR #189; the companion ruleset follow-up is reserved as a
future CI-079. Rebased onto current main.

Cross-reference: ByronWilliamsCPA/.github#188, skill-observations log #184.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit that referenced this pull request Jun 4, 2026
* feat(ci): add qlty gate and weekly health scan

Adds qlty-gate job (PR diff gate, fail-level medium) and qlty-health
job (weekly full scan, informational). No coverage upload — this is
an infra/config repo.
Refs: #188

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(ci): deny-by-default permissions and on-main SHA pin for qlty workflow

Resolve the SonarCloud gate failure on PR #189 (githubactions:S8234, MAJOR):
replace workflow-level 'permissions: read-all' with deny-by-default
'permissions: {}'. Both jobs already declare job-scoped 'contents: read', the
minimum a diff/health gate needs, matching the python-qlty-gate.yml convention.
Also flagged by Copilot.

Re-pin the reusable-workflow refs from 040026a (pre-squash feature-branch commit
of #188, diverged from main) to 1561a3e, the squash-merge commit on main that
contains python-qlty-gate.yml, so the pin is auditable and Renovate-trackable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(changelog): document repo-local qlty.yml caller workflow

Add an Unreleased entry for the qlty.yml caller added in PR #189, matching the
repo pattern where feat changes update CHANGELOG.md (as sibling PR #188 did).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit to ByronWilliamsCPA/.claude that referenced this pull request Jun 4, 2026
Adds CI-078 (suggested, override-eligible): .github/workflows/qlty.yml must
contain a qlty-gate job calling python-qlty-gate.yml. The qlty.sh GitHub App
StatusContext always reports SUCCESS regardless of blocking issue count and
cannot enforce quality gates; a real GitHub Actions CheckRun from the qlty-gate
job is required for enforcement.

Allocated CI-078 (not CI-073) to avoid an ID collision with the renovate-health
suite's CI-073..077 in PR #189; the companion ruleset follow-up is reserved as a
future CI-079. Rebased onto current main.

Cross-reference: ByronWilliamsCPA/.github#188, skill-observations log #184.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit to ByronWilliamsCPA/.claude that referenced this pull request Jun 4, 2026
Adds CI-078 (suggested, override-eligible): .github/workflows/qlty.yml must
contain a qlty-gate job calling python-qlty-gate.yml. The qlty.sh GitHub App
StatusContext always reports SUCCESS regardless of blocking issue count and
cannot enforce quality gates; a real GitHub Actions CheckRun from the qlty-gate
job is required for enforcement.

Allocated CI-078 (not CI-073) to avoid an ID collision with the renovate-health
suite's CI-073..077 in PR #189; the companion ruleset follow-up is reserved as a
future CI-079. Rebased onto current main.

Cross-reference: ByronWilliamsCPA/.github#188, skill-observations log #184.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit to ByronWilliamsCPA/rag-processor that referenced this pull request Jun 4, 2026
Adds qlty-gate job (PR diff mode, fail-level medium) and qlty-health
job (weekly full scan, informational) alongside existing coverage upload.
Refs: ByronWilliamsCPA/.github#188

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit to ByronWilliamsCPA/rag-processor that referenced this pull request Jun 4, 2026
* feat(ci): add qlty gate and weekly health scan

Adds qlty-gate job (PR diff mode, fail-level medium) and qlty-health
job (weekly full scan, informational) alongside existing coverage upload.
Refs: ByronWilliamsCPA/.github#188

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(ci): re-pin qlty-gate reusable workflow to reachable SHA

The qlty-gate and qlty-health jobs pinned
python-qlty-gate.yml@040026ab, a pre-merge commit from the
ByronWilliamsCPA/.github PR #188 branch that is not reachable from that
repo's main (the source branch was deleted on merge). GitHub Actions
cannot resolve a reusable workflow at an unreachable SHA, so qlty.yml
failed at startup on every pull_request event (0 jobs scheduled), and
the intended "qlty-gate / Qlty Gate" check never appeared.

Re-pin both jobs to 1561a3ef, the #188 merge commit on .github main,
which exposes the same input interface (fail-level, check-all, no-fail,
upstream) and the same contents: read permissions. Add a CHANGELOG
entry documenting the gate and weekly health scan.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants