Skip to content

Improve release skill from operator feedback - #4089

Merged
vmarta merged 3 commits into
developfrom
docs/release-skill-safeguards
Nov 3, 2026
Merged

vmarta merged 3 commits into
developfrom
docs/release-skill-safeguards

Conversation

@vmarta

@vmarta vmarta commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Recent releases identified gaps in the release skill. The release skill is an AI-agent runbook that guides a human operator who approves release actions. This PR applies team feedback and relevant safeguards from SFNext PR #3005 to PWA Kit.

Audit compatibility -> verify release scope -> check every published package
Verify notes and npm tags -> announce -> merge back with next-dev targets and stable MCP
  • Require read-only compatibility and scope reports. Reports include code evidence, source PR links, and verified authors. An unresolved accidental breaking change blocks a stable release.
  • Verify every selected package, including independently versioned packages. Define an approved retry process for partial publication. Verify saved GitHub release notes before continuing.
  • Protect npm tag targets for patches to older release lines. Pin generator commands to exact versions.
  • Set next-dev targets for the SDK package group, commerce-sdk-react, and retail-react-app during merge-back. Keep MCP at npm's published stable latest version; increment it only for an actual MCP release. Preserve newer develop cycles, handle same-version bumps, retain stable changelog history, and verify release ancestry.
  • Keep detailed reference information in RELEASE-INTERNALS.md. Keep actions and completion criteria in SKILL.md. This PR does not include the Storefront Next dataset or mirror automation.

Evidence

  • Before: The skill checked publication for one package and reset only the SDK version during merge-back. It did not require an explicit compatibility audit, categorized scope, saved-notes verification, or ancestry check.
  • After: Three read-only manual inspection scenarios passed. The npm same-version option check returned true. No release was performed.

Merge Danger

Door: Two-way. You can revert these documentation changes. A revert cannot undo a release action that an operator later approves and executes.

Blast Radius: Release. These changes affect AI-agent release instructions. Runtime code, publishing scripts, and CI workflows are unchanged.

How to Test

Shared prerequisites: Check out docs/release-skill-safeguards at commit 49edfc016. Work from the repository root with Node.js and npm available. Keep all checks local and read-only: do not bump versions, push, edit releases, rerun workflows, or announce a release. Treat the npm versions in Scenario 2 as hypothetical inputs; no registry query is required.

  1. Audit and scope: Open .claude/skills/release/SKILL.md at Steps 1 and 3. Then open RELEASE-INTERNALS.md in the same directory at Compatibility audit and Categorized release scope. Inspect the requirements for a patch that removes a public SDK export and for a second preview that adds a customer-visible fix.

Expect: Read-only subagents audit exact commits. Accidental breaks block stable shipment. Scope includes the added fix with a category, feature/fix classification, PR links, and verified authors.

Status: Passed by instruction inspection; delegated audit execution was not tested.

  1. Next-dev targets and stable MCP: Read Step 6 and Next-dev versions. Use shipped SDK 3.19.1, retail 10.2.0, commerce-sdk-react 5.0.1; pre-merge develop SDK 3.22.0-dev, retail 10.2.0-dev, commerce-sdk-react 6.0.0-dev, and hypothetical unshipped MCP 0.8.0-dev with npm latest at stable 0.5.0; repeat the MCP check with newly shipped 0.6.0 on latest. Run npm_config_allow_same_version=true npm config get allow-same-version and compare the instructions with scripts/bump-version/index.js and scripts/utils.js.

Expect: Targets are SDK 3.22.0-dev, retail 10.3.0-dev, commerce-sdk-react 6.0.0-dev, and stable MCP 0.5.0 (0.6.0 in the second case); bump order is SDK, commerce-sdk-react, retail, with no independent MCP bump or dev-cycle increment; the command prints true. Stable changelog history is preserved, and merge-back requires Create a merge commit plus a successful ancestry check.

Status: Passed by instruction/script inspection and command execution; no bumps or merges were performed.

  1. Publication and closeout: Read the triage table and Steps 3-5 for an SDK preview that exists while the selected commerce-sdk-react preview is missing, then for stable patch 3.19.1 when 3.21.0 is already released.

Expect: Partial publication blocks testing and post-release until a separately approved same-SHA retry and all-package verification. The older patch needs approved latest/next targets; notes require read-back and gated repair; the announcement pins generator/retail versions; closeout also requires Snyk and announcement evidence.

Status: Passed by instruction inspection; live publication and closeout were not tested.

Validation evidence and limits

A separate subagent reran all three scenarios on docs/release-skill-safeguards at 49edfc016. All passed inspection, including both stable MCP cases, and the same-version configuration command printed true. The worktree stayed clean. git diff --check passed before commit.

The results validate the instructions. They do not validate an end-to-end release. The validation did not execute actual audit delegation, GitHub author resolution, bumps, publication, tag restoration, release-note edits, merge ancestry, Snyk updates, or announcements.

@git2gus

git2gus Bot commented Nov 3, 2026

Copy link
Copy Markdown

Git2Gus App is installed but the .git2gus/config.json doesn't have right values. You should add the required configuration.

@cc-prodsec

cc-prodsec commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@vmarta
vmarta marked this pull request as ready for review November 3, 2026 16:57
@vmarta
vmarta requested a review from shethj November 3, 2026 16:57
Comment thread .claude/skills/release/SKILL.md Outdated

## Step 4 — Final release

Before publishing a patch to an older release line (for example, `3.19.1` when `3.21.0` is already released), capture existing npm `latest`/`next` tags and confirm their intended post-release targets with the operator. Stable publication moves `latest`; if a newer line must remain the default, include separately gated restoration commands in the publish plan and verify restoration before announcing. For a normal top-of-line release, both tags target the shipped stable versions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we capture the existing dist-tags before the first preview? An older-line preview moves next before this step, so the original target is lost and users are pointed to the older line. Please also plan to restore next after preview publication, as well as the appropriate tags after the stable release.

shethj
shethj previously approved these changes Nov 3, 2026
@vmarta
vmarta enabled auto-merge (squash) November 3, 2026 18:45

## Step 2 — Bump, stamp changelogs, open the PR

Run **the bump-and-PR cycle** with **preview** versions (`-preview.N`), documented in `RELEASE-INTERNALS.md`: ensure the release branch, working branch off it, run the four bumps and verify, stamp the changelogs, then **GATE** the PR into `release-<major>.<minor>.x`. You open it; the **operator merges it.** The merge triggers CI, which publishes. Announcing to teams and smoke testing both wait on that publish — they are Step 3, gated on the preview actually being on npm.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should we explicitly mention the scenario where a non version bump push is safe only when every package version already exists in npm? since the workflow still runs on any release-* push, from-package then skips existing versions, something like:

Any push to `release-*` runs the publish step. It publishes only package versions absent from npm; a non-version-bump push is a no-op only when every publish-eligible version already exists.

**First confirm the preview is actually published** — `npm view @salesforce/pwa-kit-react-sdk@<version>-preview.N version` returns the version, not a 404. Everything below depends on that version being installable; announcing or generating before it points teams at a version that doesn't exist yet.
**Publication checkpoint:** for every selected package, run `npm view @salesforce/<pkg>@<exact-version> version` and require the expected version. Use the approved release version map, including independent versions. All selected packages must be installable before announcing or generating.

If publication is partial, inspect the publishing leg for the exact merged SHA and repair its failure. **GATE** a retry of the failed workflow jobs at that same SHA/version map (`gh run rerun <run-id> --failed`), stating which missing packages may publish. Already-published versions are skipped. Recheck every selected version; keep testing and post-release blocked until all resolve.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should we clarify the status before rerunning? keeping this recovery path lerna publish from-package publishes packages individually, it has no cross package transaction or rollback, so a failure and edge case but can leave a partial publication

Because publication is per package and non-transactional, a failed run can leave some versions on npm. Recheck the full approved version map before and after the retry.

@vmarta
vmarta merged commit 0ec71e0 into develop Nov 3, 2026
41 of 42 checks passed
@vmarta
vmarta deleted the docs/release-skill-safeguards branch November 3, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants