Skip to content

Commit 88ede8e

Browse files
committed
feat(release): ship v2.0.0 annotate+draw workflow update
Promote the annotate and draw dual-capture model with redesigned compact toolbar settings UI, region-aware reporting, and MCP query support. Align landing/docs messaging and bump runtime/package versions to 2.0.0 for release readiness.
1 parent b755499 commit 88ede8e

47 files changed

Lines changed: 3373 additions & 598 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ Powered by [onLLM.dev](https://onllm.dev).
2727
- 🧩 No integration into app code
2828
- 🎛️ Per-tab ON/OFF control (off by default)
2929
- 🎯 In-page annotation dialog with intent + severity
30+
- ✍️ Draw mode for region annotations (rectangle + ellipse)
31+
- ⚙️ Compact toolbar with pop-out settings (output level + clear on copy)
3032
- 👀 Visual markers and hover targeting
3133
- 🧾 Export outputs in compact / standard / detailed / forensic formats
3234
- 🛡️ Shadow DOM isolation for stable styling
@@ -69,11 +71,13 @@ Then load it in Chrome or Edge:
6971
1. Open any supported website tab.
7072
2. Click the onUI extension icon.
7173
3. Enable `This Tab`.
72-
4. Use the on-page launcher to annotate and manage notes.
73-
5. Hold `Shift` and click multiple elements to batch-select targets.
74-
6. Release `Shift` to open a shared annotation dialog for selected targets.
75-
7. Save once to create one annotation per selected element.
76-
8. Copy exported output from the toolbar.
74+
4. Use the on-page launcher to open the compact toolbar.
75+
5. Toggle **Annotate mode** for element targeting or **Draw mode** for region targeting.
76+
6. Hold `Shift` and click multiple elements to batch-select targets.
77+
7. Release `Shift` to open a shared annotation dialog for selected targets.
78+
8. Save once to create one annotation per selected element (or one region annotation in draw flow).
79+
9. Open toolbar **Settings** to choose output level and configure **Clear on copy**.
80+
10. Copy exported output from the toolbar.
7781

7882
## 🔌 Local MCP Setup
7983

docs/mcp-setup.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,16 @@ This release supports:
99

1010
Browser support in this release: **Chrome stable + Edge stable (unpacked)**.
1111

12+
## UI Capture Model (v2.0.0)
13+
14+
This release formalizes two complementary capture flows in the extension UI:
15+
16+
1. **Annotate mode**: target individual elements (or Shift multi-select groups) for precise element-level issues.
17+
2. **Draw mode**: drag rectangle/ellipse regions for layout, spacing, and grouping issues that span multiple elements.
18+
3. **Compact toolbar + pop-out settings**: primary controls remain in a compact rail, while output level and clear-on-copy are configured in a readable settings pop-out.
19+
20+
MCP consumers receive these captures through the same local tools (`onui_get_annotations`, `onui_search_annotations`, `onui_get_report`) with region metadata preserved in report output.
21+
1222
## One-Command Setup
1323

1424
macOS/Linux:
@@ -66,6 +76,57 @@ The local MCP server exposes:
6676
7. `onui_delete_annotation`
6777
8. `onui_clear_page_annotations`
6878

79+
## Recommended v2.0.0 Agent Flow
80+
81+
1. Open the page and capture issues with **Annotate mode** for elements or **Draw mode** for regions.
82+
2. Use toolbar **Settings** to choose the output level your agent needs (`compact`, `standard`, `detailed`, `forensic`).
83+
3. Query page state through `onui_get_annotations` or generate a page-level summary with `onui_get_report`.
84+
4. Search region or element issues with `onui_search_annotations` when iterating on a subset of UI work.
85+
5. After fixes are verified, update or clear annotations through the metadata/delete tools.
86+
87+
## Region-aware MCP Query Examples
88+
89+
### Search for region annotations by shape
90+
91+
```json
92+
{
93+
"tool": "onui_search_annotations",
94+
"arguments": {
95+
"query": "region ellipse",
96+
"pageUrl": "https://example.com/ui"
97+
}
98+
}
99+
```
100+
101+
### Search by region geometry text
102+
103+
`onui_search_annotations` matches rounded region geometry tokens (`x`, `y`, `width`, `height`) in addition to comment/selector/path/tag text.
104+
105+
```json
106+
{
107+
"tool": "onui_search_annotations",
108+
"arguments": {
109+
"query": "width 120 height 80",
110+
"pageUrl": "https://example.com/ui",
111+
"status": "pending"
112+
}
113+
}
114+
```
115+
116+
### Generate a region-aware report
117+
118+
```json
119+
{
120+
"tool": "onui_get_report",
121+
"arguments": {
122+
"pageUrl": "https://example.com/ui",
123+
"level": "detailed"
124+
}
125+
}
126+
```
127+
128+
In `detailed` and `forensic` levels, region annotations include target type, shape, and geometry fields in report output.
129+
69130
## Recommended Agent Cleanup Workflow
70131

71132
If you want agents to process annotations and then remove them:

docs/release.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
# onUI Local Release Runbook
22

3+
## Release Summary (v2.0.0)
4+
5+
- Added a dual capture model with **Annotate mode** for element-level feedback and **Draw mode** for region-level layout and spacing issues.
6+
- Added draw mode shape selection (`rectangle`/`ellipse`) plus region editing UX with persistent outlines and transform handles.
7+
- Redesigned the floating toolbar to stay compact while opening output level and clear-on-copy controls in a readable pop-out settings card.
8+
- Updated report formatters (`compact`, `standard`, `detailed`, `forensic`) to emit explicit region target metadata and geometry.
9+
- Improved MCP/local search behavior so region annotations are queryable by target type/shape and geometry text, with report generation preserving region fields.
10+
- Polished toolbar/dialog behavior for mode toggles, escape handling, multi-select hints, and copy/clear settings flow.
11+
312
## Preconditions
413

514
1. Node 20+

docs/usage.md

Lines changed: 59 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,20 @@
55
1. Open the target web page.
66
2. Open the extension popup.
77
3. Toggle `This Tab` on.
8-
4. Use the floating onUI launcher to start annotating.
9-
5. Click normally to annotate a single element.
10-
6. Hold `Shift` and click multiple elements to build a batch selection.
11-
7. Release `Shift` to open one dialog for all selected targets.
12-
8. Save notes and copy output when needed.
8+
4. Use the floating onUI launcher to open the toolbar.
9+
5. Choose one annotation path:
10+
1. **Element annotation path**
11+
1. Toggle **Annotate mode**.
12+
2. Click an element to open the element dialog.
13+
3. Add comment/intent/severity and save.
14+
2. **Region draw path**
15+
1. Toggle **Draw mode**.
16+
2. Choose shape: **rectangle** or **ellipse**.
17+
3. Drag on the page to define region geometry.
18+
4. Release pointer to open the region dialog.
19+
5. Add comment/intent/severity and save.
20+
6. Optionally open **Settings** from the toolbar to choose output level and toggle **Clear on copy**.
21+
7. Copy output from the toolbar when needed.
1322

1423
## Annotation Dialog
1524

@@ -20,11 +29,51 @@ Each annotation supports:
2029

2130
## Multi-Element Batch Annotation
2231

23-
- Hold `Shift` and click to toggle elements in or out of the current batch.
24-
- Releasing `Shift` opens a single dialog with the full selected target list.
25-
- Removing targets in the dialog updates the batch before save.
26-
- Saving creates one annotation per selected element with shared internal batch metadata.
27-
- Batch selection is capped at 25 elements.
32+
1. Toggle **Annotate mode**.
33+
2. Hold `Shift` and click elements to add/remove them from the pending batch.
34+
3. Release `Shift` to open one dialog for the current connected targets.
35+
4. Optionally remove targets in the dialog before save.
36+
5. Save to create one annotation per selected element with shared batch metadata.
37+
6. Keep batch size at or below 25 elements.
38+
39+
## Region Editing
40+
41+
1. Click an existing region marker/outline to open it in edit mode.
42+
2. Move the region by dragging inside the transform box.
43+
3. Resize using the eight handles (`nw`, `n`, `ne`, `e`, `se`, `s`, `sw`, `w`).
44+
4. Save to persist updated geometry with the annotation comment/intent/severity.
45+
5. Use Delete in the region dialog to remove the region annotation.
46+
47+
## Interaction Behavior (Shift, Draw, Escape)
48+
49+
1. `Shift` multi-select applies to **element selection flow**.
50+
2. Draw mode is exclusive with annotate mode; enabling draw mode exits annotate mode.
51+
3. `Escape` behavior is state-based:
52+
1. During active draw draft: cancel current draft.
53+
2. With pending drawn region dialog: close pending region.
54+
3. In draw mode (idle): exit draw mode.
55+
4. While editing region transform geometry: clear transform edit state.
56+
5. During pending Shift multi-select (before dialog): clear pending selection.
57+
6. In annotate mode with no higher-priority state: exit annotate mode.
58+
59+
## Toolbar Settings (Compact Pop-out)
60+
61+
1. The floating toolbar remains compact by default.
62+
2. Click the **Settings** icon to open the pop-out settings card.
63+
3. Use **Output level** to switch between `compact`, `standard`, `detailed`, and `forensic` export formats.
64+
4. Use **Clear on copy** to automatically clear annotations after a successful copy action.
65+
66+
## Export and Report Output for Region Annotations
67+
68+
1. Region annotations are exported with `targetType` = `region` and include:
69+
1. `shape` (`rectangle` or `ellipse`)
70+
2. `geometry` (`x`, `y`, `width`, `height`, coordinate space)
71+
2. Output-level implications:
72+
1. **compact**: includes inline region shape + geometry summary.
73+
2. **standard**: includes target type, shape, and geometry fields.
74+
3. **detailed**: includes target type/shape/geometry plus compatibility selector/tag/path fields.
75+
4. **forensic**: includes region identification fields and region geometry in target identification.
76+
3. Region edits update geometry used in subsequent report/export output.
2877

2978
## Notes
3079

landing-page/index.html

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -63,10 +63,10 @@
6363
</a>
6464
</div>
6565

66-
<h1>Ship cleaner UI products by annotating any UI directly in the browser.</h1>
66+
<h1>Annotate elements, draw regions, and ship cleaner UI faster.</h1>
6767

6868
<p class="hero-sub">
69-
onUI adds a visual annotation layer to any web app, then exports structured context your agents can use immediately. Annotate single elements or multi-select related UI in one pass. With <code>onui-local</code> MCP connected, your agent can continuously read fresh annotations and keep applying fixes — like UI pair programming. No app code changes. No backend.
69+
onUI gives you two capture modes in one workflow: <strong>Annotate mode</strong> for element-level fixes and <strong>Draw mode</strong> for layout or spacing regions. The new compact toolbar design keeps primary actions in a minimal rail, while settings open in a focused pop-out card. With <code>onui-local</code> MCP connected, your agent can read fresh annotations and iterate continuously. No app code changes. No backend.
7070
</p>
7171

7272
<div class="hero-cta">
@@ -102,8 +102,9 @@ <h1>Ship cleaner UI products by annotating any UI directly in the browser.</h1>
102102
<div class="hero-chips">
103103
<span class="chip chip-teal">Chrome Stable</span>
104104
<span class="chip chip-teal">Edge Stable (Unpacked)</span>
105-
<span class="chip chip-teal">Multi-Element Selection</span>
106-
<span class="chip chip-blue">Local MCP</span>
105+
<span class="chip chip-teal">Annotate + multi-select</span>
106+
<span class="chip chip-teal">Draw mode regions</span>
107+
<span class="chip chip-blue">Redesigned toolbar UI</span>
107108
<span class="chip chip-amber">Zero Backend</span>
108109
</div>
109110
</section>
@@ -123,8 +124,8 @@ <h2>See onUI in action</h2>
123124

124125
<section id="features" class="section">
125126
<div class="section-header">
126-
<h2>Built for annotation-first agent workflows</h2>
127-
<p>Everything you need for AI-assisted UI development</p>
127+
<h2>Built for annotate + draw workflows</h2>
128+
<p>Element-level and region-level feedback in the redesigned compact toolbar</p>
128129
</div>
129130
<div class="feature-grid">
130131
<article class="feature-card">
@@ -138,22 +139,22 @@ <h3>No integration work</h3>
138139
<div class="feature-icon feature-icon-blue">
139140
<svg viewBox="0 0 24 24"><path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm1 15h-2v-2h2v2zm0-4h-2V7h2v6z"/></svg>
140141
</div>
141-
<h3>Precision context</h3>
142-
<p>Capture intent and severity directly on UI elements so agents understand what to fix and how urgent it is.</p>
142+
<h3>Annotate mode for precise UI fixes</h3>
143+
<p>Target individual elements, attach intent and severity, and use Shift multi-select when one issue spans multiple components.</p>
143144
</article>
144145
<article class="feature-card">
145146
<div class="feature-icon feature-icon-blue">
146147
<svg viewBox="0 0 24 24"><path fill="currentColor" d="M3 5h8v8H3V5zm10 0h8v8h-8V5zM3 15h8v4H3v-4zm10 2h8v2h-8v-2z"/></svg>
147148
</div>
148-
<h3>Multi-element selection</h3>
149-
<p>Select and annotate related elements together when one issue spans multiple UI parts.</p>
149+
<h3>Draw mode for layout and spacing</h3>
150+
<p>Drag rectangle or ellipse regions to call out spacing, alignment, grouping, and layout problems that are bigger than one element.</p>
150151
</article>
151152
<article class="feature-card">
152153
<div class="feature-icon feature-icon-amber">
153154
<svg viewBox="0 0 24 24"><path fill="currentColor" d="M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zm-5 14H7v-2h7v2zm3-4H7v-2h10v2zm0-4H7V7h10v2z"/></svg>
154155
</div>
155-
<h3>Multiple output formats</h3>
156-
<p>Export compact, standard, detailed, or forensic reports depending on your debugging depth.</p>
156+
<h3>New compact toolbar design</h3>
157+
<p>The redesigned floating rail stays small on-page, while settings open in a readable pop-out card for output level and clear-on-copy controls.</p>
157158
</article>
158159
<article class="feature-card">
159160
<div class="feature-icon feature-icon-teal">
@@ -187,8 +188,8 @@ <h2>The AI pair-programming loop</h2>
187188
<div class="workflow-grid">
188189
<div class="workflow-step workflow-step-teal">
189190
<span class="workflow-num">01</span>
190-
<h3>Mark UI issues</h3>
191-
<p>Click single elements or multi-select related UI to add annotations with intent (bug, enhancement, question) and severity.</p>
191+
<h3>Capture with Annotate or Draw</h3>
192+
<p>Use Annotate mode for single or multi-element feedback, or Draw mode to mark regions for layout and spacing issues.</p>
192193
</div>
193194
<div class="workflow-step workflow-step-blue">
194195
<span class="workflow-num">02</span>

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "onui",
3-
"version": "1.1.0",
3+
"version": "2.0.0",
44
"private": true,
55
"description": "Browser extension for annotating UI elements for AI agents",
66
"packageManager": "pnpm@9.15.0",

packages/core/package.json

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@onui/core",
3-
"version": "1.1.0",
3+
"version": "2.0.0",
44
"private": true,
55
"type": "module",
66
"main": "dist/index.js",
@@ -13,6 +13,10 @@
1313
},
1414
"scripts": {
1515
"build": "tsc",
16-
"typecheck": "tsc --noEmit"
16+
"typecheck": "tsc --noEmit",
17+
"test": "vitest run"
18+
},
19+
"devDependencies": {
20+
"vitest": "^2.1.9"
1721
}
1822
}

packages/core/src/formatters/compact.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
11
import type { Annotation, ReportContext } from '../types.js';
2-
import { truncate } from './shared.js';
2+
import { formatRegionGeometry, isRegionAnnotation, truncate } from './shared.js';
33

44
export function formatCompact(annotations: Annotation[], context: ReportContext): string {
55
const lines: string[] = [`# ${context.title}`, `URL: ${context.url}`, ''];
66

77
annotations.forEach((annotation, index) => {
8-
let line = `${index + 1}. \`${annotation.selector}\` > ${annotation.comment}`;
8+
const location = isRegionAnnotation(annotation)
9+
? `${annotation.region.shape}(${formatRegionGeometry(annotation.region.geometry)})`
10+
: `\`${annotation.selector}\``;
11+
12+
let line = `${index + 1}. ${location} > ${annotation.comment}`;
913

1014
if (annotation.selectedText) {
1115
line += ` (selected: "${truncate(annotation.selectedText, 50)}")`;

packages/core/src/formatters/detailed.ts

Lines changed: 26 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import type { Annotation, ReportContext } from '../types.js';
2-
import { truncate } from './shared.js';
2+
import { formatRegionGeometry, isRegionAnnotation, truncate } from './shared.js';
33

44
export function formatDetailed(annotations: Annotation[], context: ReportContext): string {
55
const lines: string[] = [
@@ -11,22 +11,34 @@ export function formatDetailed(annotations: Annotation[], context: ReportContext
1111
];
1212

1313
annotations.forEach((annotation, index) => {
14-
lines.push(`## ${index + 1}. ${annotation.elementPath}`);
14+
const isRegion = isRegionAnnotation(annotation);
15+
lines.push(`## ${index + 1}. ${isRegion ? `${annotation.region.shape} region` : annotation.elementPath}`);
1516
lines.push('');
16-
lines.push('### Element Info');
17-
lines.push(`- **Selector:** \`${annotation.selector}\``);
18-
lines.push(`- **Tag:** \`${annotation.tagName}\``);
19-
lines.push(`- **Path:** \`${annotation.elementPath}\``);
17+
lines.push('### Target Info');
2018

21-
if (annotation.role) {
22-
lines.push(`- **Role:** ${annotation.role}`);
23-
}
19+
if (isRegion) {
20+
lines.push('- **Target type:** `region`');
21+
lines.push(`- **Shape:** \`${annotation.region.shape}\``);
22+
lines.push(`- **Geometry:** ${formatRegionGeometry(annotation.region.geometry)}`);
23+
lines.push(`- **Selector (compat):** \`${annotation.selector}\``);
24+
lines.push(`- **Tag (compat):** \`${annotation.tagName}\``);
25+
lines.push(`- **Path (compat):** \`${annotation.elementPath}\``);
26+
} else {
27+
lines.push('- **Target type:** `element`');
28+
lines.push(`- **Selector:** \`${annotation.selector}\``);
29+
lines.push(`- **Tag:** \`${annotation.tagName}\``);
30+
lines.push(`- **Path:** \`${annotation.elementPath}\``);
2431

25-
if (annotation.attributes && Object.keys(annotation.attributes).length > 0) {
26-
lines.push('');
27-
lines.push('### Attributes');
28-
for (const [key, value] of Object.entries(annotation.attributes)) {
29-
lines.push(`- \`${key}\`: \`${truncate(value, 80)}\``);
32+
if (annotation.role) {
33+
lines.push(`- **Role:** ${annotation.role}`);
34+
}
35+
36+
if (annotation.attributes && Object.keys(annotation.attributes).length > 0) {
37+
lines.push('');
38+
lines.push('### Attributes');
39+
for (const [key, value] of Object.entries(annotation.attributes)) {
40+
lines.push(`- \`${key}\`: \`${truncate(value, 80)}\``);
41+
}
3042
}
3143
}
3244

0 commit comments

Comments
 (0)