Computer Use Skill

Desktop automation for AI agents — screenshot, click, type, scroll, accessibility trees, form filling, menu navigation, scripts (AppleScript/JXA/PowerShell), app/window management. 49 tools.

What This Skill Does

Workflow	Calls	What It Achieves
See → Click → Verify	3	Screenshot, interact, confirm
Fill Form	2	Find fields → fill all
Menu Action	2	Find shortcut → press key
Script Automation	1	Complex AppleScript/PowerShell
App Management	2	List → activate

Approach Priority

1. Keyboard shortcut — fastest (1 call)
2. Accessibility click — most reliable (click_element)
3. Script — most powerful (run_script)
4. Coordinate click — fallback (screenshot + left_click)

Installation

git clone https://github.com/zavora-ai/skill-computer-use.git \
  ~/.skills/skills/computer-use

Requirements

Required: computer-use-mcp (49 tools) Platforms: macOS (AppleScript/JXA), Windows (PowerShell) Permissions: Accessibility access in System Settings

Cross-MCP:

• mcp-browser — web automation
• worksheet-mcp — open generated reports
• mcp-slack — desktop notification context

Success Criteria

Metric	Target
Click accuracy	Correct element on first attempt
Safety	Never click without screenshot
Approach	Accessibility over coordinates

Contributors

James Karanja Maina

License

Apache-2.0 — Part of ADK-Rust Enterprise. Built with ❤️ by Zavora AI