feat: add workspace initialization and agent instructions

- Implemented initWorkspace function to set up a new workspace.
- Created renderInitSummary function to summarize initialization results.
- Added agent instructions in markdown format to guide usage of Contextful.
- Included next steps for users after workspace initialization.

Author: prasad-kumkar

README.md

@@ -1,27 +1,40 @@
 ![contextful cover image](docs/cover.svg)
 
-# contextful
+# *Contextful*
 
-**Most-efficient Context Management Layer for Agentic AI.**
+**Context Management + Search Engine for Agentic AI.**
+
+Contextful is a runtime contextual layer & local search engine for agents that gives them one fast way to find, compress, cite, and remember project context. 
+
+Available as a `cli` + MCP & Skill, it integrates seemlessly with Codex, Claude Code, Cursor, Windsurf, GitHub Copilot, VS Code, Cline, Roo Code, Continue, and Zed.
+
+<img src="docs/supported.png" alt="Contextful screenshot" width="450px" style="border-radius: 20px;" />
 
-Contextful is a runtime contextual layer agents need for real projects. Available as an MCP, it integrates with Codex, Claude Code, Cursor, Windsurf, GitHub Copilot, VS Code, Cline, Roo Code, Continue, and Zed, then gives agents one fast way to find, compress, cite, and remember project context.
 
 Instead of making an agent read 40 files every session, Contextful indexes the project once and returns a ranked, cited, token-budgeted **context pack**.
 
-## Why Agents Need This
+## Why?
+
+Context has always been a bottleneck for agentic AI. Large context window models (eg. 1m tokens) are :
+1. Expensive and require significantly more compute & processing time.
+2. Halucinate & loose key information as context window fills up. 
+3. Most projects have millions of lines of code, but agents can only fit in limited tokens per context window.
+
+The current solution is to make the agent guess which files to read, then pay the token cost to read them every session. This is slow, expensive, and lossy.
 
+Apart from this, agents have no way to store or share learnings across sessions. Every time they start, they forget everything and have to re-read the same context again.
+
+<img src="docs/context-window.png" alt="Contextful screenshot" width="450px" style="border-radius: 20px;" />
+
+I started developing Contextful to keep context window smaller by enabling efficient knowledge retrieval. We can index the project and return a ranked, cited, token-budgeted context pack, we can:
 - **100x more efficient token usage:** stop paying tokens to re-read the same files.
 - **Fewer tool calls:** one context pack can replace dozens of grep, glob, and read-file calls.
 - **No lost context between sessions:** agents can store session learnings in an evidence-backed memory ledger.
 - **Shareable project knowledge:** lessons and context packs survive context compaction and future sessions.
 
-## Core
+## Key Features
 
-### Search Engine
-
-Contextful analyzes the query, classifies intent, and combines lexical search, symbols, docs, graph relationships, and memory hits to retrieve the right evidence. The goal is Google-level project search for agents: vague queries like "resources for auth onboarding" should still land on the right code, docs, and prior lessons.
-
-### Efficient Context Storage
+### 1. Context Management
 
 The default local store is SQLite with FTS-backed search and typed graph tables. V1 ships with:
 
@@ -33,11 +46,20 @@ The default local store is SQLite with FTS-backed search and typed graph tables.
 
 The next storage upgrades are optional semantic vectors through sqlite-vec, LanceDB, or local HNSW, and compressed adjacency lists with Roaring bitmaps or CSR arrays for larger repositories.
 
-### Memory Ledger
+
+### 2. Search Engine
+
+<img src="docs/grep-vs-vector.png" alt="Contextful screenshot" width="450px" style="border-radius: 20px;" />
+
+Contextful analyzes the query, classifies intent, and combines lexical search, symbols, docs, graph relationships, and memory hits to retrieve the right evidence. The goal is Google-level project search for agents: vague queries like "resources for auth onboarding" should still land on the right code, docs, and prior lessons.
+
+
+
+### 3. Memory Ledger
 
 Agents can store lessons, decisions, and useful project facts, but not as loose "remember this" notes. Every memory requires evidence refs from files, symbols, commits, or prior context packs. When the evidence changes, Contextful marks the memory stale.
 
-### Runtime Architecture
+### 4. Contextful Execution
 
 Contextful is an MCP server, local indexer, and small CLI:
 

docs/context-window.png

@@ -0,0 +1,103 @@
+import fs from "node:fs";
+import path from "node:path";
+import { indexWorkspace } from "./indexer.js";
+import { type IndexResult } from "./types.js";
+import { relativePath, resolveWorkspace, stateDirFor } from "./util.js";
+
+export interface InitResult {
+  workspace: string;
+  stateDir: string;
+  instructionsPath: string;
+  index: IndexResult;
+  nextSteps: string[];
+}
+
+export async function initWorkspace(options: { workspace?: string }): Promise<InitResult> {
+  const workspace = resolveWorkspace(options.workspace);
+  const index = await indexWorkspace({ workspace });
+  const stateDir = stateDirFor(workspace);
+  const instructionsPath = path.join(stateDir, "AGENT_INSTRUCTIONS.md");
+  fs.writeFileSync(instructionsPath, renderAgentInstructions(workspace), "utf8");
+
+  return {
+    workspace,
+    stateDir,
+    instructionsPath,
+    index,
+    nextSteps: [
+      'Run `cxf search "where is auth handled?" --workspace .` to test retrieval.',
+      "Add `npx -y @inferensys/contextful server` to your MCP client config.",
+      "Tell the agent to call `context_pack` before broad file reads."
+    ]
+  };
+}
+
+export function renderInitSummary(result: InitResult): string {
+  const instructions = relativePath(result.workspace, result.instructionsPath);
+  return [
+    "Contextful initialized",
+    "",
+    `Workspace: ${result.workspace}`,
+    `State: ${relativePath(result.workspace, result.stateDir)}`,
+    `Agent instructions: ${instructions}`,
+    `Indexed files: ${result.index.indexedFiles}`,
+    `Chunks: ${result.index.chunks}`,
+    `Symbols: ${result.index.symbols}`,
+    `Graph edges: ${result.index.edges}`,
+    "",
+    "Next steps:",
+    ...result.nextSteps.map((step) => `- ${step}`),
+    ""
+  ].join("\n");
+}
+
+function renderAgentInstructions(workspace: string): string {
+  return `# Contextful Agent Instructions
+
+Workspace: ${workspace}
+
+Use Contextful as the first retrieval layer for this project. The goal is to avoid broad file-reading loops and return cited, token-budgeted evidence before making code changes.
+
+## Default Rule
+
+Before running broad grep/glob/read-file exploration, call:
+
+\`\`\`text
+context_pack(query, budget, scope)
+\`\`\`
+
+Use a budget between 800 and 3000 tokens for normal tasks. Use a higher budget only when the user asks for a deep architecture pass.
+
+## When To Use Contextful
+
+- Vague project questions: "where is auth handled", "resources for billing webhooks", "how does onboarding state work".
+- Architecture questions: flows, dependencies, config paths, module relationships, or tests.
+- Impact questions before editing shared code.
+- Memory questions where prior session learnings might matter.
+- Any task where you are about to read more than 3 files to gather context.
+
+## Tool Preference
+
+1. \`context_pack\` for most questions.
+2. \`search_code\` for narrow exact lookup after the pack.
+3. \`trace_path\` or \`impact_analysis\` when relationships matter.
+4. \`recall_memory\` before repeating known project work.
+5. \`write_lesson\` only when the lesson has valid evidence refs.
+
+## Evidence Rules
+
+- Treat citations from Contextful as the source of truth for follow-up reads.
+- If a claim will affect code, verify it against the cited files before editing.
+- Do not write memory without evidence refs from files, symbols, commits, or prior context packs.
+- If evidence changes or appears stale, prefer fresh search over old memory.
+
+## Local CLI For Humans
+
+\`\`\`bash
+cxf index --workspace .
+cxf search "where is auth handled" --workspace . --budget 2000
+cxf memory add --workspace . --claim "..." --evidence file:src/example.ts:1-20
+cxf server
+\`\`\`
+`;
+}