The improve-aeo-geo/skill.md states that robots.txt must allow 9 specific AI crawlers (line 49):
robots.txt must NOT block these 9 AI crawlers: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, anthropic-ai, ChatGPT-User, Bytespider, CCBot.
The plain-text robots.txt example at lines 53-85 correctly lists all 9. However, the Next.js robots.ts code example at lines 327-341 only includes 4 specific bots plus a wildcard:
rules: [
{ userAgent: '*', allow: '/' },
{ userAgent: 'GPTBot', allow: '/' },
{ userAgent: 'ClaudeBot', allow: '/' },
{ userAgent: 'PerplexityBot', allow: '/' },
{ userAgent: 'Google-Extended', allow: '/' },
],Missing from the Next.js example:
OAI-SearchBotanthropic-aiChatGPT-UserBytespiderCCBot
Why This Matters
The wildcard * rule with Allow: / does cover these bots in practice, so the generated robots.txt will functionally work. However, the skill's own audit check (line 47-49) specifically looks for explicit rules for all 9 bots. A user following the Next.js example literally would pass the functional test but might fail the audit if it checks for explicit user-agent entries.
More importantly, the inconsistency between examples undermines confidence — a user comparing the plain-text version against the Next.js version will wonder which is correct.
Fix
Add the missing 5 bot entries to the Next.js robots.ts example to match the plain-text version.
The
improve-aeo-geo/skill.mdstates that robots.txt must allow 9 specific AI crawlers (line 49):The plain-text robots.txt example at lines 53-85 correctly lists all 9. However, the Next.js
robots.tscode example at lines 327-341 only includes 4 specific bots plus a wildcard:Missing from the Next.js example:
OAI-SearchBotanthropic-aiChatGPT-UserBytespiderCCBotWhy This Matters
The wildcard
*rule withAllow: /does cover these bots in practice, so the generated robots.txt will functionally work. However, the skill's own audit check (line 47-49) specifically looks for explicit rules for all 9 bots. A user following the Next.js example literally would pass the functional test but might fail the audit if it checks for explicit user-agent entries.More importantly, the inconsistency between examples undermines confidence — a user comparing the plain-text version against the Next.js version will wonder which is correct.
Fix
Add the missing 5 bot entries to the Next.js
robots.tsexample to match the plain-text version.