diff --git a/drafts/2026-05-04T023705Z.md b/drafts/2026-05-04T023705Z.md new file mode 100644 index 0000000..b01f0d5 --- /dev/null +++ b/drafts/2026-05-04T023705Z.md @@ -0,0 +1,109 @@ +# Reply draft: LLMs-as-abstraction blast-radius subthread, tool-boundary clamp + +**Status:** draft (pending manual post) + +**HN:** https://news.ycombinator.com/item?id=47999520 +Parent comment (replying to): https://news.ycombinator.com/item?id=48002997 + +**Story / OP:** "LLMs Are Not a Higher Level of Abstraction" (lelanthran's blog, +posted by lelanthran, 54 points, 8 hours old, 48 comments). Article argues LLMs +are not a real abstraction layer in the way compilers / web frameworks / CMSes +are, because you can't reach below them and reason about what's happening. + +**Discovery:** found via /news front page (rank ~18) + cross-checked on +hn.algolia.com search. Picked because the distalx subthread on this thread +specifically pivots into concrete tool-layer blast-radius examples +("silently delete your production database", "open a security backdoor"), +which is exactly the seam FailProof addresses. Subthread is fresh (last reply +within the past hour at draft time), engagement still active. + +**Parent comment (distalx, id=48002997, depth 40):** + +> A transmission error has a strictly contained, predictable blast radius. +> If a packet drops, the system knows exactly how to handle it: it throws +> a timeout, drops a connection, or asks for a retry. The worst-case +> scenario is known. +> +> A reasoning error has an infinite, unpredictable blast radius. When an +> LLM hallucinates, it doesn't fail safely but it writes perfectly +> compiling code that does the wrong thing. That "wrong thing" might just +> render a button incorrectly, or it might silently delete your production +> database, or open a security backdoor. +> +> You can build reliable abstractions over failures that are predictable +> and contained. You cannot abstract away unpredictable destruction. + +(Sibling reply by yunwal, id=48003219, says "It's quite easy to limit the +blast radius of a reasoning error" with no specifics; amazingamazing, +id=48003529, follows up with "How so?" Both are still open. Replying to +the distalx root rather than yunwal's terse line so the engagement is with +the actual argument, not the throwaway counter.) + +**My reply:** + +``` +(disclosure: I work on FailProof AI: https://github.com/exospherehost/failproofai) + +The "infinite blast radius" framing is right at the reasoning layer +and wrong at the tool boundary. The model can decide anything, but +the side effect only happens when something downstream actually runs +the decision. That gap between "decided" and "executed" is the +abstraction seam you build into. + +For "silently delete your production database" specifically: a +tool-call hook that pattern-matches DROP / TRUNCATE / DELETE FROM +before the SQL leaves the box turns "silent wipe" into "agent gets +a deny and has to reroute". warn-destructive-sql in our own ruleset +is a few lines of regex doing exactly this. The hard part isn't +writing the check, it's having a hook surface to attach it to. +Claude Code, Codex, and the Anthropic Agents SDK expose one; Cursor +and OpenCode are getting there. +``` + +**Insight for the FailProof team:** + +The "abstraction seam" framing - "the model decides anything, the tool +boundary is where you bound it" - is the cleanest one-line pitch I've +written for FailProof's positioning. It directly answers the most common +objection ("LLMs are non-deterministic, so any guardrail is just whack-a-mole"): +no, the LLM is non-deterministic but the tool call is a deterministic event +you can intercept before side effect. Worth lifting into the docs landing +page or a blog post titled "where the abstraction actually is in agent +stacks", because every variant of distalx's argument we see on HN is the +same shape and the same answer applies. + +Also: distalx mentions "open a security backdoor" as a second example. That +one is harder to gate at a single tool-call boundary (the backdoor manifests +across multiple non-destructive edits) and we currently don't have a good +policy for it. Worth thinking about whether some kind of cross-call diff +review or "introduce-new-network-surface" detector belongs in the built-in +set. Filing as a product thought. + +**Notes / findings:** + +- Thread is 8 hours old, 48 comments, 54 points. distalx's subthread is the + most concrete on the page; the rest is mostly compiler-vs-LLM analogy + arguments. Reply targets the live concrete subthread, not the OP article + (the article is at the model/abstraction layer, which is the wrong gate + per CLAUDE.md "Threads where the parent's pain is at the model layer ... + No"; the subthread isn't). +- ASCII-only punctuation throughout: no em / en dashes, no curly quotes, no + ellipses, straight slashes for the SQL-keyword list. +- Single named policy (warn-destructive-sql), single concrete failure + (silently delete prod DB), single disclosure link, no install command, no + feature catalog, no dashboard mention. Matches the "working shape" from + comments/2026-04-29T043958Z.md, not the flagged shape from + drafts/2026-05-01T184439Z.md. +- One nuance: I use "in our own ruleset" rather than "FailProof has a + warn-destructive-sql policy" so the policy mention reads as incidental + rather than as a feature plug. This came out of the post-mortem rule + about marketing-cadence connectives. +- Reply is at depth 80 (one level under distalx). Visibility is fine since + the OP is fresh; if posting after the thread cools, drop it instead. +- Cross-thread duplicate check (CLAUDE.md rule 7): grepped drafts/ + + comments/ + open PRs for item?id=47999520 - no hit. Body is unique to + this thread (no paraphrase of any prior FailProof draft body in the + repo). +- Did not target the saturated front-page "Agentic Coding Is a Trap" + (id=48002442) or "DeepClaude" (id=48002136) - the first is a meta vent + thread per the gate, the second is at the model-swap layer.