Skip to content

Extend /versions skill: toolkit-wide staleness check + render-baseline regression tests #23

@ConalMullan

Description

@ConalMullan

Problem

The /versions skill detects mismatches within a user's project (npx remotion versions) and toolkit-vs-GitHub-release drift, but two gaps prevent it from being the keystone for routine version maintenance:

  1. No toolkit-wide staleness signal. Templates can be pinned consistently but still N months behind upstream Remotion (currently 4.0.425 pinned vs 4.0.464 latest — 39 patches, ~3 months). The skill doesn't surface this.
  2. No rendering verification. Without a way to detect that a Remotion bump silently changed render output, every version bump requires manual eyeball verification by the maintainer — a single point of failure for a 1.3k-star public toolkit.

Proposed extensions

1. Staleness check (small)

Compare templates/*/package.json pinned versions against npm view remotion version. Output as part of standard /versions invocation:

Toolkit pins: 4.0.425
Upstream:     4.0.464  (39 patches, 3 months behind)

2. Render-baseline regression check (the unlock)

  • tests/baseline-composition.tsx — fixed composition exercising representative APIs (interpolate, Sequence, OffthreadVideo, Audio, TransitionSeries, calculateMetadata)
  • Committed baseline PNGs (frames 0/30/60 at --scale=0.25)
  • scripts/render-baseline.mjs — renders + pixel-diffs vs baselines
  • CI job triggered on PRs touching package.json with Remotion changes; posts visual diff as PR comment

3. --bump-toolkit maintainer mode

Walks templates one at a time:

  • npx remotion upgrade
  • Render baseline → diff against checked-in baseline
  • Report per-template ("matches" / "differs — review needed")
  • Generates structured output for the bump PR

Why this matters

  • Unblocks Dependabot for remotion/@remotion/* packages
  • Makes it safe to accept contributor PRs that bump Remotion
  • Removes the maintainer as the QA bottleneck
  • Visible "we test renders" signal to OSS users

Scope

Out of scope for this issue: actually migrating templates to @remotion/media (deliberate separate decision per .claude/skills/remotion/SKILL.md).

Effort estimate: ~1-2 days for someone familiar with Remotion + the existing /versions skill.

Background

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions