Skip to content

feat: add plugin verify evidence endpoint#2411

Draft
dylanzhangzx wants to merge 2 commits into
openclaw:mainfrom
dylanzhangzx:plugin-verify-evidence
Draft

feat: add plugin verify evidence endpoint#2411
dylanzhangzx wants to merge 2 commits into
openclaw:mainfrom
dylanzhangzx:plugin-verify-evidence

Conversation

@dylanzhangzx

@dylanzhangzx dylanzhangzx commented May 27, 2026

Copy link
Copy Markdown

Summary

This PR adds a plugin-side verification evidence endpoint:

  • GET /api/v1/plugins/{name}/verify
  • version/tag/latest resolution matching the existing package release model
  • machine-readable release evidence for plugin identity, publisher identity, artifact metadata, provenance/source linkage, trusted publisher metadata, security scan status, moderation/download blocking, compatibility, and capabilities
  • schema export, generated schema dist output, HTTP API documentation, and trust-semantics spec

This is intended as a small incremental evidence surface for plugin trust review. It does not add a vetted-community badge, does not change official status semantics, and does not introduce a generated plugin-card.md artifact yet.

Why

#2272 discusses a public review path for vetted community plugins. Before ClawHub can decide whether to expose a future plugin trust tier or badge, users and maintainers need a stable version-scoped evidence surface for plugin releases.

Skills now have Skill Card / verify surfaces. Plugins already expose package security, artifact, readiness, source-linked metadata, and trusted publisher data through separate surfaces. This endpoint collects the plugin-specific evidence into one machine-readable envelope without making any endorsement decision.

Shape

A successful response includes:

  • schema: "clawhub.plugin.verify.v1"
  • ok, decision, and reasons
  • plugin identity and publisher identity
  • selected version, resolution source, and creation time
  • review status (official or unreviewed-community)
  • artifact metadata and file hashes
  • source/provenance/trusted publisher evidence
  • compact security and scanner signals
  • compatibility and capability metadata

Trust semantics

The endpoint is documented in specs/plugin-verify.md.

  • Community packages remain review.status: "unreviewed-community" unless they are already official.
  • ok: true only means the selected release is not blocked, has an effectively clean ClawScan status, and does not have a stale trust summary.
  • ok: true does not mean the plugin is official, endorsed, manually reviewed, or safe for every deployment.
  • Future vetted-community or trust-card work should add a separate review signal rather than reusing ok.

Real behavior proof

Local handler request against a mocked package release:

GET https://clawhub.local/api/v1/plugins/@openclaw/example-plugin/verify?version=1.0.5
status 200

Response excerpt:

{
  "schema": "clawhub.plugin.verify.v1",
  "ok": true,
  "decision": "pass",
  "reasons": [],
  "name": "@openclaw/example-plugin",
  "displayName": "Example Plugin",
  "family": "code-plugin",
  "publisherHandle": "example",
  "version": "1.0.5",
  "resolvedFrom": "version",
  "review": {
    "status": "unreviewed-community",
    "isOfficial": false,
    "channel": "community"
  },
  "artifact": {
    "kind": "npm-pack",
    "sha256": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
    "format": "tgz",
    "files": [
      {
        "path": "openclaw.plugin.json",
        "contentType": "application/json"
      }
    ]
  },
  "provenance": {
    "tier": "source-linked",
    "scope": "artifact-only",
    "sourceRepo": "openclaw/example-plugin",
    "sourceCommit": "4d515c",
    "hasProvenance": true,
    "trustedOpenClawPlugin": false,
    "trustedPublisher": {
      "provider": "github-actions",
      "repository": "openclaw/example-plugin",
      "workflowFilename": "publish.yml"
    },
    "source": "source-linked-release"
  },
  "security": {
    "status": "clean",
    "blockedFromDownload": false,
    "pending": false,
    "stale": false
  },
  "compatibility": {
    "pluginApiRange": ">=2026.3.24",
    "builtWithOpenClawVersion": "2026.5.24"
  },
  "capabilities": {
    "executesCode": true,
    "hooks": ["before_dispatch"],
    "capabilityTags": ["security", "hook:before-dispatch"]
  },
  "signature": {
    "status": "unsigned"
  }
}

Observed result: the endpoint returns version-scoped plugin evidence, keeps the community plugin explicitly unreviewed, preserves source/provenance fields, and does not imply an official endorsement.

What was not tested: a live production ClawHub deployment, UI rendering, signed verification documents, or a future vetted-community review tier.

Tests

  • bunx vitest run convex/httpApiV1.handlers.test.ts --testNamePattern "plugin verify|package security endpoint returns exact release trust"
  • bun run format:check
  • bunx tsc -p packages/schema/tsconfig.json --noEmit
  • bunx tsc --noEmit
  • bun run lint

@vercel

vercel Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Someone is attempting to deploy a commit to the Amantus Machina Team on Vercel.

A member of the Team first needs to authorize it.

@clawsweeper

clawsweeper Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 20, 2026, 1:57 AM ET / 05:57 UTC.

Summary
The PR adds a public plugin verification JSON endpoint, schema export, generated schema output, docs, spec text, and handler coverage.

Reproducibility: not applicable. this is a feature/API proposal, not a report of broken current-main behavior. The available output is mocked handler evidence, not a live backend reproduction path.

Review metrics: 2 noteworthy metrics.

  • Diff Size: +532/-1 across 8 files. The change spans backend route code, schema exports, generated dist, docs, tests, and a security-sensitive spec.
  • Public API Surface: 1 GET endpoint added. A new trust-related JSON contract may be consumed by install, review, or automation clients.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Make trusted-publisher/provenance evidence release-scoped or explicitly label it as current package configuration.
  • [P1] Add redacted live HTTP proof against a running ClawHub/Convex backend with fixture package state.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body and tests show mocked handler output only; before merge it needs redacted live HTTP output or logs from a running ClawHub/Convex setup, with private details such as IPs, API keys, phone numbers, and non-public endpoints redacted. After adding proof, updating the PR body should trigger a fresh ClawSweeper review; if not, a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The response can imply release-time trusted-publisher assurance for older selected versions even though trustedPublisher is current package configuration.
  • [P1] The broader vetted-community plugin trust RFC is still open, so clients could automate against fields whose long-term trust semantics are not maintainer-approved.
  • [P1] The PR body still lacks redacted live HTTP output or logs from a running ClawHub/Convex setup.

Maintainer options:

  1. Make Provenance Release-Scoped (recommended)
    Move trusted-publisher data out of release provenance or persist and return only evidence that was true for the selected artifact at publish time.
  2. Accept Current-Config Semantics Explicitly
    Maintainers could keep the field only if the schema and docs clearly label it as current package configuration, not selected-release provenance.
  3. Pause Behind The Trust RFC
    Leave this draft open until the vetted-community plugin trust RFC settles the public review signal and assurance boundaries.

Next step before merge

  • [P1] The blockers are maintainer trust/security contract judgment and contributor real-runtime proof rather than a safe autonomous repair lane.

Security
Needs attention: The diff introduces a trust-boundary concern because current package trusted-publisher config is exposed as version-scoped provenance evidence.

Review findings

  • [P1] Keep trusted-publisher evidence release-scoped — convex/httpApiV1/packagesV1.ts:698
Review details

Best possible solution:

Keep the endpoint proposal open, separate release-scoped evidence from current package configuration, and land it only after maintainer trust-contract approval plus real HTTP proof.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a feature/API proposal, not a report of broken current-main behavior. The available output is mocked handler evidence, not a live backend reproduction path.

Is this the best way to solve the issue?

No. A plugin verify endpoint is a plausible layer, but this shape still mixes current package trusted-publisher configuration into a version-scoped provenance envelope.

Full review comments:

  • [P1] Keep trusted-publisher evidence release-scoped — convex/httpApiV1/packagesV1.ts:698
    trustedPublisher is populated from current package-level config inside a version-scoped provenance response. If that config is added, changed, or revoked after an older release, /verify?version=... can report assurance the selected artifact did not have; use release-time evidence or clearly move/label this as current package configuration outside release provenance.
    Confidence: 0.88

Overall correctness: patch is incorrect
Overall confidence: 0.88

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6f28659e7bfb.

Label changes

Label justifications:

  • P2: This is a meaningful public API and plugin trust-surface feature with limited runtime blast radius but important security semantics.
  • merge-risk: 🚨 compatibility: The PR adds a new public plugin API response contract that clients may automate against.
  • merge-risk: 🚨 security-boundary: The response exposes install trust, scanner, provenance, and review semantics for executable plugin releases.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body and tests show mocked handler output only; before merge it needs redacted live HTTP output or logs from a running ClawHub/Convex setup, with private details such as IPs, API keys, phone numbers, and non-public endpoints redacted. After adding proof, updating the PR body should trigger a fresh ClawSweeper review; if not, a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

Security concerns:

  • [medium] Trusted-publisher evidence is not release-scoped — convex/httpApiV1/packagesV1.ts:698
    A public verify response may guide install trust decisions, so showing current trusted-publisher metadata under release provenance can overstate the assurance of older selected versions.
    Confidence: 0.86

What I checked:

  • repository-policy-read: AGENTS.md was read fully; its guidance on security-sensitive API trust boundaries, specs, and real Convex validation applies to this PR. (AGENTS.md:1, 6f28659e7bfb)
  • current-main-lacks-plugin-verify: Current main has skill verify surfaces but no plugin verify schema, route, or docs entry, so this PR is not obsolete as implemented on main. (6f28659e7bfb)
  • pr-adds-public-route: The PR routes GET /api/v1/plugins/{name}/verify, resolves a selected release, and then fetches the package trusted-publisher record before building the response. (convex/httpApiV1/packagesV1.ts:3326, f0c3ad45cd7c)
  • trusted-publisher-under-provenance: The PR places toPublicTrustedPublisher(params.trustedPublisher) inside the version-scoped provenance object. (convex/httpApiV1/packagesV1.ts:698, f0c3ad45cd7c)
  • version-scope-contract: The new spec says verification is version-scoped and marks changes to ok, decision, reasons, or review.status as security-sensitive because install clients may automate on them. (specs/plugin-verify.md:55, f0c3ad45cd7c)
  • trusted-publisher-is-current-package-config: Current main stores trusted publishers in a package-level table keyed by packageId, while releases store separate publishActor and verification metadata. (convex/schema.ts:1765, 6f28659e7bfb)

Likely related people:

  • Patrick-Erichsen: Blame and log history show recent work on skill verify, package trust schema, and current package HTTP API surfaces. (role: recent trust/API surface contributor; confidence: high; commits: 5be7035a691c, e3e5705d8969, b62d8ca81345; files: convex/httpApiV1/packagesV1.ts, convex/httpApiV1/skillsV1.ts, convex/schema.ts)
  • Onur: Commit history links package trusted publishing via GitHub OIDC to the trusted-publisher table and publish path used by this PR. (role: introduced trusted publishing behavior; confidence: medium; commits: 8592272720fe; files: convex/schema.ts, convex/packages.ts, convex/httpApiV1/packagesV1.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels May 27, 2026
@clawsweeper

clawsweeper Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@dylanzhangzx

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please update it or it will be closed.

@github-actions github-actions Bot added the stale label Jun 4, 2026
@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 4, 2026
@github-actions github-actions Bot removed the stale label Jun 5, 2026
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 5, 2026
@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels Jun 11, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 11, 2026
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator

Hey @dylanzhangzx , is this still something you're interested in getting merged? If so just reply and I'll take a look. Thanks for opening!

@dylanzhangzx

Copy link
Copy Markdown
Author

Yes, still interested. The goal is to provide a plugin-side evidence surface that can support the broader plugin trust/review discussion in #2272, without implying official endorsement.

I agree the trustedPublisher/provenance semantics need care. I’m happy to revise the response shape so release-scoped evidence is separated from current package-level trusted publisher configuration, and add real HTTP proof against a running setup before moving this out of draft.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants