Skip to content

Commit e5c8256

Browse files
sealdayclaude
sealday
and
claude
authored
refactor(aumate): remove GUI module and add Electron support (#14)
* refactor(aumate): remove GUI module and add Electron support BREAKING CHANGE: Remove GUI-related features from aumate Task 1: Remove GUI from aumate - Remove gui, click_helper, window_manager features - Delete 100+ GUI source files (gui/, screenshot/, clipboard_manager/, etc.) - Remove GUI dependencies (egui, wgpu, winit, muda, tray-icon, rfd, etc.) - Update bot package to remove GUI bindings - Update botjs to remove GUI exports - Delete 8 GUI example files Task 2: Add Electron support for @tego/bot - Create platform-specific npm packages (darwin-x64, darwin-arm64, win32-x64-msvc, etc.) - Add optionalDependencies to main package - Update CI/CD workflow to publish platform packages 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(ci): add missing validation for index.d.ts before copying The workflow validated index.js exists but not index.d.ts, which could lead to unclear error messages if the TypeScript declaration file was missing from the artifact. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs(botjs): regenerate API documentation and add image-match tests - Add PixelColor type export for hex color strings - Add private options field to ScreenshotTool class - Fix JSDoc warnings for unused @PARAM annotations - Add vitest alias for proper coverage tracking - Add comprehensive unit tests for image-match module - Regenerate TypeDoc API documentation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(ci): properly handle npm publish errors for platform packages The previous error handling masked all failures with a generic warning. Now it distinguishes between expected "already exists" errors (which are skipped) and genuine failures (auth, network, validation) which correctly fail the build. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent b192537 commit e5c8256

280 files changed

Lines changed: 2869 additions & 32881 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/release.yml‎

Lines changed: 109 additions & 88 deletions
Original file line numberDiff line numberDiff line change
@@ -62,17 +62,10 @@ jobs:
6262
clang \
6363
pkg-config \
6464
libssl-dev \
65-
libglib2.0-dev \
66-
libgtk-3-dev \
67-
libayatana-appindicator3-dev \
68-
libwayland-dev \
6965
libxcb1-dev \
7066
libxrandr-dev \
7167
libdbus-1-dev \
7268
libpipewire-0.3-dev \
73-
libegl1-mesa-dev \
74-
libgles2-mesa-dev \
75-
libgbm-dev \
7669
libxi-dev \
7770
libxtst-dev \
7871
libxdo-dev \
@@ -85,29 +78,27 @@ jobs:
8578
- name: Build NAPI binary
8679
run: |
8780
cd packages/bot
88-
pnpm napi build --release --esm --const-enum --output-dir dist --platform --target ${{ matrix.settings.target }}
81+
pnpm napi build --release --esm --const-enum --platform --target ${{ matrix.settings.target }}
8982
shell: bash
9083

9184
- name: List built files
92-
run: ls -la packages/bot/dist/
85+
run: ls -la packages/bot/
9386
shell: bash
9487

9588
- name: Upload artifact
9689
uses: actions/upload-artifact@v5
9790
with:
9891
name: bindings-${{ matrix.settings.target }}
9992
path: |
100-
packages/bot/dist/*.node
101-
packages/bot/dist/index.js
102-
packages/bot/dist/index.d.ts
93+
packages/bot/*.node
94+
packages/bot/index.js
95+
packages/bot/index.d.ts
10396
if-no-files-found: error
10497

105-
publish-npm:
106-
name: Publish to NPM
98+
publish-platform-packages:
99+
name: Publish Platform Packages
107100
runs-on: ubuntu-latest
108-
needs:
109-
- build-napi
110-
101+
needs: build-napi
111102
steps:
112103
- name: Checkout code
113104
uses: actions/checkout@v4
@@ -118,14 +109,6 @@ jobs:
118109
node-version: "20"
119110
registry-url: "https://registry.npmjs.org"
120111

121-
- name: Setup pnpm
122-
uses: pnpm/action-setup@v4
123-
with:
124-
version: 8
125-
126-
- name: Install dependencies
127-
run: pnpm install -r
128-
129112
- name: Download all artifacts
130113
uses: actions/download-artifact@v4
131114
with:
@@ -136,58 +119,98 @@ jobs:
136119
echo "-- Downloaded artifacts structure --"
137120
ls -laR artifacts/
138121
139-
- name: Create dist directory for bot
140-
run: mkdir -p packages/bot/dist
141-
142-
- name: Move all .node binaries to bot dist
122+
- name: Prepare and publish platform packages
143123
run: |
144-
echo "-- Moving .node files --"
145-
for artifact_dir in artifacts/bindings-*; do
124+
VERSION=$(node -p "require('./packages/bot/package.json').version")
125+
echo "Package version: $VERSION"
126+
127+
# Map target to platform package
128+
declare -A TARGET_MAP
129+
TARGET_MAP["x86_64-apple-darwin"]="darwin-x64"
130+
TARGET_MAP["aarch64-apple-darwin"]="darwin-arm64"
131+
TARGET_MAP["x86_64-pc-windows-msvc"]="win32-x64-msvc"
132+
TARGET_MAP["i686-pc-windows-msvc"]="win32-ia32-msvc"
133+
TARGET_MAP["aarch64-pc-windows-msvc"]="win32-arm64-msvc"
134+
TARGET_MAP["x86_64-unknown-linux-gnu"]="linux-x64-gnu"
135+
136+
for target in "${!TARGET_MAP[@]}"; do
137+
platform="${TARGET_MAP[$target]}"
138+
artifact_dir="artifacts/bindings-$target"
139+
npm_dir="packages/bot/npm/$platform"
140+
141+
echo "Processing $target -> $platform"
142+
146143
if [ -d "$artifact_dir" ]; then
147-
target=$(basename "$artifact_dir" | sed 's/^bindings-//')
148-
echo "Processing target: $target"
144+
# Find the .node file
145+
node_file=$(find "$artifact_dir" -name "*.node" -type f | head -1)
149146
150-
node_file=$(find "$artifact_dir" -name "*.node" -type f)
151147
if [ -n "$node_file" ]; then
152-
case "$target" in
153-
x86_64-apple-darwin)
154-
dest_name="bot.darwin-x64.node"
155-
;;
156-
aarch64-apple-darwin)
157-
dest_name="bot.darwin-arm64.node"
158-
;;
159-
x86_64-pc-windows-msvc)
160-
dest_name="bot.win32-x64-msvc.node"
161-
;;
162-
i686-pc-windows-msvc)
163-
dest_name="bot.win32-ia32-msvc.node"
164-
;;
165-
aarch64-pc-windows-msvc)
166-
dest_name="bot.win32-arm64-msvc.node"
167-
;;
168-
x86_64-unknown-linux-gnu)
169-
dest_name="bot.linux-x64-gnu.node"
170-
;;
171-
*)
172-
echo "Warning: Unknown target $target, using generic name"
173-
dest_name="bot.$target.node"
174-
;;
175-
esac
176-
177-
echo " Copying: $node_file -> packages/bot/dist/$dest_name"
178-
cp -v "$node_file" "packages/bot/dist/$dest_name"
148+
# Determine correct binary name
149+
dest_name="bot.$platform.node"
150+
echo " Copying: $node_file -> $npm_dir/$dest_name"
151+
cp -v "$node_file" "$npm_dir/$dest_name"
152+
153+
# Update version in package.json
154+
cd "$npm_dir"
155+
npm version "$VERSION" --no-git-tag-version --allow-same-version || true
156+
157+
# Publish the platform package
158+
echo " Publishing @tego/bot-$platform@$VERSION"
159+
if ! publish_output=$(npm publish --access public 2>&1); then
160+
if echo "$publish_output" | grep -qiE "(already exists|previously published|cannot publish over)"; then
161+
echo " Skipping: Package version already exists"
162+
else
163+
echo " Error: Failed to publish platform package"
164+
echo "$publish_output"
165+
exit 1
166+
fi
167+
else
168+
echo " Published successfully"
169+
fi
170+
cd - > /dev/null
179171
else
180172
echo " Warning: No .node file found in $artifact_dir"
181173
fi
174+
else
175+
echo " Warning: Artifact directory $artifact_dir not found"
182176
fi
183177
done
184-
echo "-- Final dist contents --"
185-
ls -la packages/bot/dist/
178+
env:
179+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
180+
181+
publish-npm:
182+
name: Publish to NPM
183+
runs-on: ubuntu-latest
184+
needs:
185+
- build-napi
186+
- publish-platform-packages
187+
188+
steps:
189+
- name: Checkout code
190+
uses: actions/checkout@v4
191+
192+
- name: Setup Node.js
193+
uses: actions/setup-node@v4
194+
with:
195+
node-version: "20"
196+
registry-url: "https://registry.npmjs.org"
197+
198+
- name: Setup pnpm
199+
uses: pnpm/action-setup@v4
200+
with:
201+
version: 8
202+
203+
- name: Install dependencies
204+
run: pnpm install -r
205+
206+
- name: Download all artifacts
207+
uses: actions/download-artifact@v4
208+
with:
209+
path: artifacts
186210

187-
- name: Copy NAPI index files from artifact
211+
- name: Copy NAPI index files for main package
188212
run: |
189213
# Copy index.js and index.d.ts from x86_64-unknown-linux-gnu artifact
190-
# These files are the same for all platforms
191214
artifact_dir="artifacts/bindings-x86_64-unknown-linux-gnu"
192215
193216
if [ ! -d "$artifact_dir" ]; then
@@ -197,45 +220,31 @@ jobs:
197220
198221
if [ ! -f "$artifact_dir/index.js" ]; then
199222
echo "Error: index.js not found in $artifact_dir"
200-
echo "Contents of $artifact_dir:"
201-
ls -la "$artifact_dir" || echo "Directory is empty or doesn't exist"
223+
ls -la "$artifact_dir" || echo "Directory is empty"
202224
exit 1
203225
fi
204226
205227
if [ ! -f "$artifact_dir/index.d.ts" ]; then
206228
echo "Error: index.d.ts not found in $artifact_dir"
207-
echo "Contents of $artifact_dir:"
208-
ls -la "$artifact_dir" || echo "Directory is empty or doesn't exist"
229+
ls -la "$artifact_dir" || echo "Directory is empty"
209230
exit 1
210231
fi
211232
212233
echo "Copying index files from $artifact_dir"
213-
cp -v "$artifact_dir/index.js" packages/bot/dist/
214-
cp -v "$artifact_dir/index.d.ts" packages/bot/dist/
215-
216-
# Verify both files were copied successfully
217-
if [ ! -f packages/bot/dist/index.js ]; then
218-
echo "Error: Failed to copy index.js"
219-
exit 1
220-
fi
221-
if [ ! -f packages/bot/dist/index.d.ts ]; then
222-
echo "Error: Failed to copy index.d.ts"
223-
exit 1
224-
fi
234+
cp -v "$artifact_dir/index.js" packages/bot/
235+
cp -v "$artifact_dir/index.d.ts" packages/bot/
225236
226237
echo "✓ Both index.js and index.d.ts copied successfully"
227-
ls -lh packages/bot/dist/index.*
238+
ls -lh packages/bot/index.*
228239
shell: bash
229240

230241
- name: Verify bot package structure
231242
run: |
232-
echo "-- Bot package dist/ contents --"
233-
ls -la packages/bot/dist/
243+
echo "-- Bot package contents --"
244+
ls -la packages/bot/
234245
echo "-- Checking for required files --"
235-
test -f packages/bot/dist/index.js && echo "✓ index.js found" || echo "✗ index.js missing"
236-
test -f packages/bot/dist/index.d.ts && echo "✓ index.d.ts found" || echo "✗ index.d.ts missing"
237-
echo "-- .node files --"
238-
find packages/bot/dist/ -name "*.node" -exec basename {} \;
246+
test -f packages/bot/index.js && echo "✓ index.js found" || echo "✗ index.js missing"
247+
test -f packages/bot/index.d.ts && echo "✓ index.d.ts found" || echo "✗ index.d.ts missing"
239248
240249
- name: Build TypeScript packages
241250
run: pnpm ts:build
@@ -288,11 +297,23 @@ jobs:
288297
pnpm add @tego/botjs@${{ steps.version.outputs.VERSION }}
289298
```
290299
300+
### Electron Support
301+
The package now supports Electron out of the box with prebuilt binaries.
302+
Platform-specific packages are automatically installed via optionalDependencies.
303+
291304
### Packages
292305
- `@tego/bot@${{ steps.version.outputs.VERSION }}` - Rust core with N-API bindings
293306
- `@tego/botjs@${{ steps.version.outputs.VERSION }}` - TypeScript wrapper (recommended)
294307
- `@tego/bot-agent@${{ steps.version.outputs.VERSION }}` - AI-powered CLI tool
295308
309+
### Platform Binaries
310+
- `@tego/bot-darwin-x64` - macOS Intel
311+
- `@tego/bot-darwin-arm64` - macOS Apple Silicon
312+
- `@tego/bot-win32-x64-msvc` - Windows x64
313+
- `@tego/bot-win32-ia32-msvc` - Windows x86
314+
- `@tego/bot-win32-arm64-msvc` - Windows ARM64
315+
- `@tego/bot-linux-x64-gnu` - Linux x64 (glibc)
316+
296317
### What's Changed
297318
See the [CHANGELOG](https://github.com/tegojs/bot/blob/main/CHANGELOG.md) for details.
298319

‎CLAUDE.md‎

Lines changed: 7 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ This is a **monorepo** containing two major projects:
1313
- `@tego/bot` - Rust core with N-API bindings
1414
- `@tego/botjs` - TypeScript wrapper with full type safety
1515
- `@tego/bot-agent` - AI-powered CLI for script generation
16-
- `aumate` - Cross-platform Rust automation library with GUI support
16+
- `aumate` - Cross-platform Rust automation library
1717

1818
### 2. Aumate App - Tauri Desktop Application
1919
**Aumate App** is a desktop automation application built with Tauri, React, and TypeScript. It implements **Domain-Driven Design (DDD)** architecture with strict layering.
@@ -58,15 +58,8 @@ The Node.js automation packages use a **monorepo workspace** structure:
5858
- Full TypeScript type definitions
5959
- Enhanced screen capture utilities
6060
- Convenience functions for common automation tasks
61+
- Image template matching
6162
- Comprehensive test coverage
62-
- **GUI Widgets** (via `@tego/botjs` gui module):
63-
- Basic: `label`, `button`, `textInput`, `checkbox`, `slider`, `progressBar`, `separator`, `spacer`
64-
- Layout: `hbox`, `vbox`, `grid`
65-
- Containers: `panel`, `scrollArea`, `group`
66-
- Advanced: `dropdown`, `radioGroup`, `textArea`, `tabs`, `image`
67-
- Interactive: `link`, `selectableLabel`, `dragValue`, `colorPicker`, `hyperlink`, `hyperlinkUrl`, `imageButton`
68-
- **File Dialog Functions**: `showOpenFileDialog`, `showSaveFileDialog`, `showFolderDialog`
69-
- **Font Functions**: `getSystemFonts` - returns sorted list of system font families
7063

7164
### 3. `packages/bot-agent` - AI-Powered CLI (`@tego/bot-agent`)
7265
- **Language**: TypeScript (ESM)
@@ -83,42 +76,19 @@ The Node.js automation packages use a **monorepo workspace** structure:
8376

8477
### 4. `packages/aumate` - Desktop Automation Library (`aumate`)
8578
- **Language**: Rust 2024 edition (requires Rust 1.85+)
86-
- **Purpose**: Cross-platform desktop automation library with GUI support
79+
- **Purpose**: Cross-platform desktop automation library
8780
- **Features** (configurable via Cargo features):
88-
- `input` - Mouse and keyboard control via `enigo` and `rdev`
81+
- `input` - Mouse and keyboard control via `enigo`
8982
- `screen` - Screen capture via `xcap`
9083
- `clipboard` - Text and image clipboard operations via `arboard`
9184
- `window` - Window management via `active-win-pos-rs`
92-
- `gui` - Floating window system with effects (includes screen and clipboard)
93-
- **Key dependencies**:
94-
- `winit` for window management
95-
- `wgpu` for GPU rendering
96-
- `egui` / `egui-wgpu` / `egui-winit` for UI
97-
- `muda` for menu support
98-
- `tray-icon` for system tray
99-
- `rfd` for native file dialogs
100-
- `font-kit` for system font enumeration
101-
- `noise` for procedural effects
102-
- `glam` for math types
85+
- `image_match` - Image template matching via `imageproc`
10386
- **Module structure**:
10487
- `input/` - Mouse and keyboard control (implements `FromStr` for button parsing)
10588
- `screen.rs` - Screen capture operations
10689
- `clipboard.rs` - Clipboard operations
10790
- `window.rs` - Window management
108-
- `gui/` - GUI sub-modules:
109-
- `window/` - Window controller, floating windows, configuration
110-
- `effect/` - Particle effects system with 18 presets
111-
- `animation/` - Animation system with easing functions
112-
- `content/` - Content rendering (image, text)
113-
- `menu_bar/` - System tray and menu bar support
114-
- `screenshot/` - Screenshot mode with selection and toolbar:
115-
- `plugins/` - Plugin system (save, copy, cancel)
116-
- `renderer/` - WGPU/egui rendering
117-
- `ui/` - Toolbar components
118-
- **18 Particle Effects**: Rotating Halo, Pulse Ripple, Flowing Light, Stardust Scatter, Electric Spark, Smoke Wisp, Aurora Wave, Cosmic Strings, Heartbeat Pulse, Laser Beam, Lightning Arc, Matrix Rain, Meteor Shower, Orbit Rings, Rain Drop, Silk Ribbon, Sonar Pulse, Fire Glow
119-
- **Animations**: fade, scale, slide, bounce, rotate, blink with easing functions
120-
- **25 GUI Widgets**: Label, Button, TextInput, Checkbox, Slider, ProgressBar, Image, Separator, Spacer, HBox, VBox, Grid, Panel, ScrollArea, Group, Dropdown, RadioGroup, TextArea, Tabs, Link, SelectableLabel, DragValue, ColorPicker, Hyperlink, ImageButton
121-
- **19 Widget Events**: button_click, text_changed, text_submit, checkbox_changed, slider_changed, focus_gained, focus_lost, mouse_enter, mouse_leave, selection_changed, radio_changed, tab_changed, link_clicked, selectable_label_changed, drag_value_changed, color_changed, hyperlink_clicked, file_dialog_completed, font_changed
91+
- `image_match/` - Image template matching engine
12292

12393
### Aumate App DDD Architecture
12494

@@ -200,9 +170,6 @@ cargo build -p aumate
200170

201171
# Build aumate with specific features
202172
cargo build -p aumate --no-default-features --features "input,screen"
203-
204-
# Run aumate demo (screenshot mode)
205-
cargo run -p aumate
206173
```
207174

208175
### Aumate App - Build
@@ -297,14 +264,6 @@ pnpm ex:run screenshot-basic # Basic screenshot
297264
pnpm ex:run screenshot-advanced # Advanced screenshot operations
298265
pnpm ex:run screenshot-color-picker # Color picker from screen
299266
pnpm ex:run window # Window management
300-
pnpm ex:run gui-hello # Simple GUI window
301-
pnpm ex:run gui-form # Form with inputs
302-
pnpm ex:run gui-events # GUI event handling
303-
pnpm ex:run gui-notification # Notification windows
304-
pnpm ex:run gui-widgets # Dropdown, RadioGroup, TextArea, Tabs
305-
pnpm ex:run gui-interactive-widgets # Link, SelectableLabel, DragValue, ColorPicker, Hyperlink, ImageButton
306-
pnpm ex:run gui-file-dialogs # Native file open/save/folder dialogs
307-
pnpm ex:run gui-font-picker # System font enumeration and picker
308267
```
309268

310269
### Documentation
@@ -457,5 +416,5 @@ The AI is instructed to:
457416
- **Documentation**:
458417
- Tego Bot: VitePress + TypeDoc (`pnpm docs:dev`)
459418
- Aumate App: See `crates/docs/` for architecture details
460-
- **Context7 MCP**: Use for version-specific docs lookup (egui, wgpu, winit, etc.)
419+
- **Context7 MCP**: Use for version-specific docs lookup
461420
- **Roadmap**: See `docs/developments/aumate-roadmap.md`

0 commit comments

Comments
 (0)