[Test Improver] Monthly Activity 2026-05

[danielmeppiel]

🤖 Test Improver here - I'm an automated AI assistant focused on improving tests for this repository.

Activity for May 2026

Suggested Actions for Maintainer

Comprehensive list of all pending actions requiring maintainer attention (excludes items already actioned and checked off).

• Review PR [Test Improver] tests(compile): add unit tests for compile/cli.py helper functions #1206: tests(compile): 17 unit tests for compile/cli.py helper functions - Review
• Review PR [Test Improver] tests(compile): add unit tests for compile() command early-exit paths #1233: tests(compile): 13 unit tests for compile() command early-exit paths - Review
• Review PR [Test Improver] tests(install): add unit tests for integration template module #1287: tests(install): 13 unit tests for integration template module - Review
• Review PR [Test Improver] tests(cache): add unit tests for cache.integrity SHA verification #1302: tests(cache): 20 unit tests for cache.integrity SHA verification - Review
• Close PR [Test Improver] tests: skip runtime factory tests when LLM not installed #1139: changes superseded by main (runtime factory tests already fixed in main) - View

(If no actions needed, state "No suggested actions at this time.")

Maintainer Priorities

No specific priorities communicated yet. Note: PRs #1029 (ANSI fix) and #1059 (uninstall engine tests) were closed by maintainer on 2026-05-03 without merging. Be selective about PRs.

Testing Opportunities Backlog

1. compile-cli-full (medium) - compile() main command body; helper functions covered (PR [Test Improver] tests(compile): add unit tests for compile/cli.py helper functions #1206); early-exit paths covered (PR [Test Improver] tests(compile): add unit tests for compile() command early-exit paths #1233); full compilation flow still largely untested
2. deps-cli-update (medium) - src/apm_cli/commands/deps/cli.py; update() orchestration partially uncovered
3. policy-extends-regression (medium) - src/apm_cli/policy/inheritance.py + policy_checks.py; dependencies.require layering via extends: org just fixed in fix(policy): layer dependencies.require through extends: org (closes #1201) #1290; regression tests for multi-level inheritance edge cases would be valuable
4. integration-test-markers (medium) - 8 network-gated integration test files need pytestmark annotations; offered help 2026-05-09, no maintainer response
5. compile-watcher (low) - src/apm_cli/commands/compile/watcher.py ~5% coverage (file-watching, hard to test)
6. policy-status-ascii (low) - 7 tests failing due to Rich unicode (PR [Test Improver] Fix 7 failing tests: strip ANSI codes in _ascii_only helper #1029 closed by maintainer - do not retry)

Discovered Commands

# Install dev dependencies (use python3.12 to avoid PyPy)
pip install uv && python -m uv sync --extra dev --python /usr/bin/python3.12

# Run unit tests
.venv/bin/pytest tests/unit/ -q

# Lint / format
.venv/bin/ruff check src/ tests/ --fix && .venv/bin/ruff format src/ tests/

Notes: Use .venv/bin/ruff not uv run ruff. Baseline 2026-05-13: 7 failing (7 policy_status ANSI), 8287 passed.

Run History

2026-05-14 06:51 UTC - Run

• [i] Task 2: Reviewed recent commits (2026-05-11 to 2026-05-14); policy extends fix (fix(policy): layer dependencies.require through extends: org (closes #1201) #1290) and marketplace auth fix (fix: marketplace install auth host on *.ghe.com (closes #1285) #1292) both shipped with tests
• [i] Task 2: Added policy-extends-regression to backlog - multi-level inheritance edge cases still worth covering
• [i] Task 4: PR [Test Improver] tests(cache): add unit tests for cache.integrity SHA verification #1302 (cache-integrity) confirmed open as draft; all 5 TI PRs healthy
• [i] Task 7: Updated monthly issue; holding off new PRs given 5 already awaiting review

2026-05-13 06:51 UTC - Run

• [+] Task 3: 20 new tests for cache/integrity.py (SHA verification across all git layouts). Branch: test-assist/cache-integrity-tests (PR [Test Improver] tests(cache): add unit tests for cache.integrity SHA verification #1302)
• [i] Task 4: PRs [Test Improver] tests(compile): add unit tests for compile/cli.py helper functions #1206, [Test Improver] tests(compile): add unit tests for compile() command early-exit paths #1233, [Test Improver] tests(install): add unit tests for integration template module #1287 all in good shape; [Test Improver] tests: skip runtime factory tests when LLM not installed #1139 still dirty/superseded
• [i] Baseline unchanged: 7 failing (policy_status ANSI only), 8287 passing

2026-05-12 06:47 UTC - Run

• [+] Task 3: 13 new tests for install/template.py (run_integration_template + _integrate_materialization). Branch: test-assist/install-template-tests
• [*] Task 4: PR [Test Improver] tests: skip runtime factory tests when LLM not installed #1139 merge conflict - runtime_factory fix already in main; commented suggesting closure
• [i] Task 4: PRs [Test Improver] tests(compile): add unit tests for compile/cli.py helper functions #1206, [Test Improver] tests(compile): add unit tests for compile() command early-exit paths #1233 CI-green, waiting for review - no action needed
• [i] Updated baseline: 7 failing (policy_status ANSI only), 8287 passing

2026-05-11 06:57 UTC - Run

• [*] Task 4: PR [Test Improver] tests(deps): add unit tests for deps CLI helper functions #1155 was merged by danielmeppiel on 2026-05-08 (15 unit tests for deps CLI helpers)
• [*] Task 4: PRs [Test Improver] tests: skip runtime factory tests when LLM not installed #1139, [Test Improver] tests(compile): add unit tests for compile/cli.py helper functions #1206, [Test Improver] tests(compile): add unit tests for compile() command early-exit paths #1233 all CI-green, no action needed
• [i] No new human comments on Audit unmarked integration tests + swap test-integration.sh to pytest discovery (PR2 of #1166) #1168 since 2026-05-09 offer; held off re-engaging
• [i] Task 7: Updated this monthly issue

2026-05-10 06:44 UTC - Run

• [+] Task 3: 13 new tests for compile() command early-exit paths (PR [Test Improver] tests(compile): add unit tests for compile() command early-exit paths #1233). Branch: test-assist/compile-command-early-exit-tests
• [i] Discovered: 3 previously-ignored test files now all pass. Updated test command.
• [i] Task 7: Updated this monthly issue

2026-05-09 06:34 UTC - Run

• [*] Task 4: PRs [Test Improver] tests: skip runtime factory tests when LLM not installed #1139, [Test Improver] tests(compile): add unit tests for compile/cli.py helper functions #1206 both fully CI-green, no action needed
• [i] Task 5: Commented on Audit unmarked integration tests + swap test-integration.sh to pytest discovery (PR2 of #1166) #1168 (offered to add pytestmark annotations to 8 network-gated integration files)
• [i] Task 5: Commented on [FEATURE] Extract skip_download logic into testable helpers #552 (offered to implement Option B mocked integration tests for skip_download logic)
• [i] Task 7: Updated this monthly issue

2026-05-08 06:26 UTC - Run

• [*] Task 4: PRs [Test Improver] tests: skip runtime factory tests when LLM not installed #1139, [Test Improver] tests(deps): add unit tests for deps CLI helper functions #1155 both CI-green, no action needed
• [+] Task 3: 17 new tests for compile/cli.py helpers. Branch: test-assist/compile-cli-helper-tests
• [!] Observed: new test_skill_bundle.py YAML failures (8+) in main - not caused by TI
• [i] Task 7: Updated this monthly issue

2026-05-06 03:36 UTC - Run

• [*] Task 4: PR [Test Improver] tests: skip runtime factory tests when LLM not installed #1139 CI-green, no action needed
• [i] Task 2: Identified deps/cli.py helpers as 0%-covered; compile/cli.py at 49%, watcher.py at 5%
• [+] Task 3: Added 15 tests for deps CLI helper functions. Branch: test-assist/deps-cli-helper-tests
• [i] Task 7: Updated this monthly issue

2026-05-05 03:36 UTC - Run

• [+] Task 3: Fixed 4 CI failures in test_runtime_factory.py. Branch: test-assist/fix-runtime-factory-skip-tests
• [i] Discovered: marketplace tests now pass (106->7 failures). Remaining 7 are policy_status ANSI issue.
• [i] Task 7: Updated this monthly issue

2026-05-04 03:36 UTC - Run

• [!] Task 2: Observed 106 test failures -- 86+ from marketplace test drift, 7 from ANSI policy_status, 4 from runtime_factory
• [i] Task 7: Updated this monthly issue

2026-05-03 03:37 UTC - Run

• [+] Task 3: Fix logic-replay anti-pattern in TestExplicitTargetDirCreation (Test gap: TestExplicitTargetDirCreation and TestContentHashFallback are logic-replays #768). Branch: test-assist/fix-logic-replay-tests-768
• [i] Task 7: Updated this monthly issue

2026-05-02 03:35 UTC - Run

• [+] Task 3: Added 7 tests for view.py error paths. Branch: test-assist/view-error-handling-tests
• [i] Task 7: Created this monthly issue

2026-05-01 03:36 UTC - Run

• [i] Task 4: Verified PR [Test Improver] test: add unit tests for uninstall engine helpers #1059 and PR [Test Improver] Fix 7 failing tests: strip ANSI codes in _ascii_only helper #1029 (CI passing)
• [i] Task 7: Closed 2026-04 issue, created 2026-05 issue [Test Improver] Monthly Activity 2026-05 #1087

Generated by Daily Test Improver · ● 846.4K · ◷

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/daily-test-improver.md@b87234850bf9664d198f28a02df0f937d0447295

[github-actions]

Triage decision

accept

Proposed labels

area/testing
type/automation
status/accepted

Milestone

null

Suggested next action

Review PR #1059 (uninstall engine tests) and PR #1029 (ANSI fix), then close the stale older Test Improver PRs (#977, #1011) that are superseded.

Suggested issue comment

Thanks for this monthly activity snapshot. The test coverage work tracked here -- especially the install-command and compile-cli-full gaps -- aligns with APM's quality goals as the command surface grows.

Action items from this report:

- [*] Review PR #1059 (21 uninstall engine tests) -- CI is green, ready to merge.
- [*] Review PR #1029 (ANSI fix, 7 tests) -- CI passing, supersedes #977 and #1011.
- [*] Close stale PRs #977 and #1011 as superseded by #1029.
- [*] Review or close older Test Improver PRs (#430, #498, #522, #859, #890, #927, #943) -- these are from prior months and may be stale.

The test backlog (install-command ~43%, compile-cli-full ~38%) is a good target for the next automated cycle. No blocking concerns.

Per-lens notes (collapsed)

DevX UX Expert -- User-Need Reviewer

This is a monthly tracking issue generated by the Daily Test Improver agentic workflow. It aggregates pending PR reviews, stale PR cleanup candidates, and a test-coverage backlog. No CLI command surfaces, flags, or user-facing flows are impacted. The requested actions are maintainer-internal quality infrastructure work. Valid; no UX concerns.

Supply Chain Security Expert -- Risk-Surface Reviewer

No risk-surface implication. The tracked PRs (#1059, #1029) touch only test files and the _ascii_only utility helper. The testing backlog items (install-command, compile-cli-full, deps-cli) add coverage to existing code paths without touching lockfile logic, auth resolution, download paths, or MCP configuration. No theme/security or theme/governance label required.

OSS Growth Hacker -- Contributor-Tone Reviewer

Not activated -- author is a COLLABORATOR with established interactions on this repository; first-time contributor tone tuning would not meaningfully improve the reply.

Python Architect -- Architecture Reviewer

Not activated -- this is a monthly activity tracker aggregating test improvement work; no cross-cutting design decision, interface change, or new module is proposed or required before code can land.

Doc Writer -- Documentation Reviewer

Not activated -- test coverage improvements do not require user-facing documentation changes in docs/src/content/docs/ or the README.

APM CEO -- Triage Arbiter

Monthly test-coverage tracking is legitimate and continuous quality infrastructure aligned with APM's maturity goals. The PRs listed are real and review-ready. Accept the tracker; no milestone is appropriate for a perpetual monthly rhythm. Labels: area/testing + type/automation + status/accepted. No theme/* (pure CI/test infra with no product surface implication). No priority label (housekeeping cadence). Reply is brief and action-confirming.

{
  "decision": "accept",
  "decision_detail": "",
  "theme": null,
  "areas": ["area/testing"],
  "type": "type/automation",
  "status": "status/accepted",
  "priority": null,
  "preserved_labels": ["automation", "testing"],
  "milestone": null,
  "next_action": "Review PR #1059 (uninstall engine tests) and PR #1029 (ANSI fix), then close the stale older Test Improver PRs (#977, #1011) that are superseded.",
  "comment_markdown": "Thanks for this monthly activity snapshot. The test coverage work tracked here -- especially the install-command and compile-cli-full gaps -- aligns with APM's quality goals as the command surface grows.\n\nAction items from this report:\n\n- [*] Review PR #1059 (21 uninstall engine tests) -- CI is green, ready to merge.\n- [*] Review PR #1029 (ANSI fix, 7 tests) -- CI passing, supersedes #977 and #1011.\n- [*] Close stale PRs #977 and #1011 as superseded by #1029.\n- [*] Review or close older Test Improver PRs (#430, #498, #522, #859, #890, #927, #943) -- these are from prior months and may be stale.\n\nThe test backlog (install-command ~43%, compile-cli-full ~38%) is a good target for the next automated cycle. No blocking concerns."
}

---

Triage status: agentic proposal pending human ratification.
Silence is approval. Maintainers can:

• Override any label or milestone above by editing it directly --
  human edits are authoritative and will not be reverted on
  subsequent runs.
• Re-trigger triage by applying the status/needs-triage label, or
  by removing status/triaged to enroll the issue in the next
  daily sweep.

Posted by the Triage Panel workflow. See .apm/skills/apm-triage-panel for the panel skill.

Note

🔒 Integrity filter blocked 40 items

The following items were blocked because they don't meet the GitHub integrity level.

• [FEATURE] Extended MCP Server Configuration in apm.yml #20 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Allow install to download and unpack a GitHub release automatically #514 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Skills from .well-known URIs #554 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• Feature: per-target agent frontmatter transformation during deployment #581 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• bug: apm run start picks wrong runtime — system PATH stub takes priority over APM-managed binary, codex v0.116+ incompatible with GitHub Models #605 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Add ability to specify a different configuration file name or path #612 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• bug: apm install plugin@marketplace validation bypasses registry proxy #615 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Renovate supporting apm.yml #639 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] MCP install target for Claude Code (project + user scope) #643 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• Align CONTRIBUTING, Python formatting (Black/isort), and CI #645 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Copilot CLI user-scope instructions: can apm install -g write copilot-instructions.md #650 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• Marketplace add source parity with the Anthropic spec #676 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] PEP 8 Support #690 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] URL-based marketplace work Extension #692 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Create root AGENTS.md documenting all CI-enforced rules for coding agents #695 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Kiro Support #702 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• ... and 24 more items

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by Triage Panel · ● 820K · ◷