Skip to content

Add include rules for short system prompt fragments#731

Open
Liyux3 wants to merge 1 commit into
Piebald-AI:mainfrom
Liyux3:fix/extract-short-prompts
Open

Add include rules for short system prompt fragments#731
Liyux3 wants to merge 1 commit into
Piebald-AI:mainfrom
Liyux3:fix/extract-short-prompts

Conversation

@Liyux3
Copy link
Copy Markdown

@Liyux3 Liyux3 commented May 10, 2026

Summary

Several system prompt fragments in Claude Code's cli.js are under 500 characters and get filtered out by the minLength check in validateInput(). This means users cannot customize these prompts via tweakcc's system prompt editing feature.

These fragments come from:

  • Doing Tasks section (rJ3 function): exploratory questions behavior, code comment policy, UI testing, prefer-editing rule, no-gold-plating rule
  • Tone and Style section (HM3 function): emoji policy, concise output, colon-before-tool-calls rule
  • Memory section: "What NOT to save in memory" guidelines

Changes

Added specific string-match include rules in validateInput(), following the same pattern already used for the security testing prompt (line 28-30). No changes to the filtering logic itself.

Test plan

  • Run node tools/promptExtractor.js against Claude Code 2.1.126 binary and verify the newly included prompts appear in the output
  • Check that no false positives are introduced (the matches are specific substring checks)

Summary by CodeRabbit

  • Bug Fixes
    • Enhanced input validation to accept additional shorter prompt fragments from multiple content sections, improving flexibility in accepted input formats.

Review Change Stack

Several system prompt fragments in Claude Code's cli.js are under 500
characters and get filtered out by the minLength check in validateInput().
These include prompts from the Doing Tasks section (rJ3 function), Tone
and Style section (HM3 function), and the memory What NOT to Save section.

This adds specific string-match include rules following the existing
pattern used for the security testing prompt, so these short fragments
are extracted into the prompts database.

Affected prompts:
- "For exploratory questions..." (Doing tasks)
- "Don't explain WHAT the code does..." (Doing tasks)
- "For UI or frontend changes..." (Doing tasks)
- "Prefer editing existing files..." (Doing tasks)
- "Default to writing no comments..." (Doing tasks)
- "Don't add features, refactor..." (Doing tasks)
- "Only use emojis..." (Tone and style)
- "Your responses should be short and concise." (Tone and style)
- "Do not use a colon before tool calls..." (Tone and style)
- "What NOT to save in memory" (Memory instructions)
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 10, 2026

📝 Walkthrough

Walkthrough

The validateInput function in tools/promptExtractor.js adds allowlist checks for specific short text snippets from prompt sections ("Doing tasks", "Tone and style", "What NOT to save"), enabling them to bypass minimum-length and heuristic rejection filters while preserving existing Git-status and system-reminder inclusion logic.

Changes

Prompt Validation Allowlist Enhancement

Layer / File(s) Summary
Validation Allowlist Logic
tools/promptExtractor.js
New text.includes and text.startsWith checks for whitelisted prompt fragments from "Doing tasks", "Tone and style", and "What NOT to save" sections are evaluated before length and heuristic filters to permit these short snippets as valid inputs.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

  • Piebald-AI/tweakcc#341: Modifies validateInput with heuristic whitelist/blacklist checks for specific prompt text snippets in the same file.
  • Piebald-AI/tweakcc#400: Updates promptExtractor.js to whitelist short system-reminder text that corresponds to prompt updates.

Suggested reviewers

  • bl-ue

Poem

🐰 A rabbit hops through validation gates,
With whitelists that seal the fates,
Short snippets now dance free and light,
Bypassing filters left and right,
From "Doing tasks" to "What NOT save," 🎉
Validation's now a little brave.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and specifically describes the main change—adding include rules for short system prompt fragments—which directly matches the core objective of the PR to enable extraction of previously filtered-out short prompt sections.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tools/promptExtractor.js (1)

32-47: ⚡ Quick win

Use section separators instead of new prose comments

Line 32, Line 40, and Line 45 add new prose comments. This conflicts with the repo rule to avoid adding comments unless explicitly requested. If grouping is needed, use the // ====== style section separator instead.

As per coding guidelines, "Do not add comments unless explicitly requested" and "Use // ====== lines for major code sections".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/promptExtractor.js` around lines 32 - 47, The added prose comments
before the short-prompt blocks violate the repo rule; replace those freeform
comments with section separator lines (e.g., // ====== Short prompts from the
"Doing tasks" section) rather than prose, keeping the existing conditional
checks on the text variable (the blocks related to rJ3 and HM3 in cli.js and the
"What NOT to save" memory checks) intact and without extra sentences.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tools/promptExtractor.js`:
- Line 38: Replace the double-quoted string in the conditional using
text.startsWith with a single-quoted literal and escape the internal apostrophe
so it conforms to the project's Prettier style; specifically update the
expression text.startsWith("Don't add features, refactor") to use single quotes
and an escaped apostrophe (i.e., text.startsWith('Don\'t add features,
refactor')) in tools/promptExtractor.js where the startsWith check is performed.

---

Nitpick comments:
In `@tools/promptExtractor.js`:
- Around line 32-47: The added prose comments before the short-prompt blocks
violate the repo rule; replace those freeform comments with section separator
lines (e.g., // ====== Short prompts from the "Doing tasks" section) rather than
prose, keeping the existing conditional checks on the text variable (the blocks
related to rJ3 and HM3 in cli.js and the "What NOT to save" memory checks)
intact and without extra sentences.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 076d89c0-fe08-499f-92b8-c8384c4f3615

📥 Commits

Reviewing files that changed from the base of the PR and between 8a6ea7a and cfbc35f.

📒 Files selected for processing (1)
  • tools/promptExtractor.js

Comment thread tools/promptExtractor.js
skrabe added a commit to skrabe/tweakcc-fixed that referenced this pull request May 14, 2026
…lter exemption, ~75 include rules

Closes the gap between the public extractor (caught 75/293 of Piebald's
2.1.140 named prompts) and what's reachable in cli.js. Six surgical
changes; on 2.1.141 cli.js they push our output from 214 prompts (75
named) to 324 prompts (322 named) including 41 net-new finds Piebald
doesn't have.

1. Unicode decode (decodeUnicodeEscapesInPiece): \uHHHH / \u{X+} / \xHH
   decoded in TemplateLiteral raw pieces. Preserves \\ so literal
   "—" source isn't accidentally interpreted. Aligns our pieces
   with Piebald's stored format so fuzzy carryover matches.

2. Fuzzy fingerprint carryover in mergeWithExisting: first-100-char
   prefix match with quote-escape normalization, collisions dropped.
   Catches prompts whose content shifted between versions (the merge
   used to require exact pieces+identifiers equality).

3. Markdown allow-list: text >= 300 chars starting with `# / ## / ###`
   header bypasses the English-keyword heuristic. Catches markdown
   skills (Managed Agents, Anthropic CLI) whose first 500 chars lack
   "you/ai/must/should/always".

4. Tool/agent/skill directive allow-list: patterns like `Use this
   {tool,skill,agent}`, `Your strengths:`, `<system-reminder>`, plus
   nine specific 400-500c openings (Find elements, Ultraplan handoff,
   ultrareview help, ...).

5. Subset filter exemption for ${...} interpolated values: keeps inner
   prompts standalone (general-purpose agent's ${"You are..."},
   conditional `You are an interactive agent ${...}` variants).

6. 65 short-prompt include rules (covers PR Piebald-AI#731 fragments + 55 more)
   and 3 block rules (HTML doctype, claude-help CLI text, JSON-schema
   "When true, ... on the API" config docs).
skrabe added a commit to skrabe/tweakcc-fixed that referenced this pull request May 14, 2026
324 prompts, 322 named. Cuts to the 75/293 baseline our extractor
produced before this round of work — see the preceding extractor and
systemPromptSync commits for the mechanics.

This release includes 41 net-new finds the in-tree extractor now
catches that Piebald's published 2.1.140 didn't have: the Task tool
descriptions (TaskUpdate/List/Get), the conditional intro variants,
plan-mode Phase 4, the new claude-code-guide agent description,
Message Batches API TypeScript reference, settings-file-locations
section, code-review wrap-in-analysis-tags, the autonomous loop tick
(dynamic pacing) variant, and the PR Piebald-AI#731 short-prompt fragments.

README target-versions line bumped 2.1.140 → 2.1.141.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant