diff --git a/BATCH_HIGHLIGHT_FEATURE.md b/BATCH_HIGHLIGHT_FEATURE.md new file mode 100644 index 0000000..049d20a --- /dev/null +++ b/BATCH_HIGHLIGHT_FEATURE.md @@ -0,0 +1,303 @@ +# 🎯 批量高亮 + 选择器测试功能说明 + +## 📋 功能概述 + +**版本**: v2.4.0 +**发布日期**: 2025-11-10 + +新增"批量高亮"和"选择器类型选择"功能，允许用户： +1. 一键高亮所有提取的元素或当前分类的元素 +2. 选择不同的选择器类型进行测试 +3. 查看实时统计反馈（成功/失败/不可用） + +## ✨ 核心特性 + +### 1. 选择器类型选择 +支持 9 种选择器类型，用户可以自由切换测试： + +| 选择器类型 | 可靠性 | 说明 | +|-----------|--------|------| +| 🎯 智能优先 | - | 7级回退逻辑（默认） | +| 🟢 Extract ID | 最高 | data-extract-id 属性 | +| 🟢 CSS by ID | 高 | #element-id | +| 🟡 XPath Short | 中 | 优化的 XPath | +| 🟡 XPath Full | 中 | 完整 XPath 路径 | +| 🟡 CSS Combined | 中 | 组合选择器 | +| 🟡 CSS by Class | 中 | .class-name | +| 🟡 CSS by Attribute | 中 | [name="value"] | +| 🔴 CSS nth-child | 低 | :nth-child(n) | + +### 2. 批量高亮操作 + +#### ✨ 高亮所有元素 +- 一次性高亮页面上所有提取的元素 +- 每个元素独立的覆盖层和动画 +- 10 秒后自动清除（比单个元素的 5 秒更长） + +#### 🎯 高亮当前分类 +- 只高亮当前标签页的元素 +- 例如：只高亮所有按钮、只高亮所有链接等 + +#### ❌ 清除高亮 +- 手动清除所有高亮覆盖层 +- 清空统计信息 + +### 3. 实时统计反馈 + +高亮操作后，会显示详细的统计信息： +``` +高亮结果：成功 45/200 | 无此选择器 120 | 定位失败 35 +``` + +- **成功数/总数**: 成功高亮的元素数量 +- **无此选择器**: 元素没有该类型的选择器（例如没有 ID） +- **定位失败**: 有选择器但无法在页面上定位到元素 + +## 🎨 UI 设计 + +### 界面布局 + +``` +┌─────────────────────────────────────┐ +│ 📊 提取统计 │ +│ [总计: 200] [页面: xxx] [时间: xxx] │ +├─────────────────────────────────────┤ +│ 🔍 搜索框 │ +├─────────────────────────────────────┤ +│ 选择器类型: [🎯 智能优先 ▼] │ +│ │ +│ ⚠️ 高亮结果：成功 45/200 | ... │ +│ │ +│ [✨ 高亮所有] [🎯 当前分类] [❌ 清除]│ +├─────────────────────────────────────┤ +│ [按钮 50] [链接 80] [表单 10] ... │ +├─────────────────────────────────────┤ +│ 元素列表... │ +└─────────────────────────────────────┘ +``` + +### 按钮样式 +- **✨ 高亮所有**: 粉色到橙色渐变，全宽 +- **🎯 当前分类**: 紫色边框，轮廓样式 +- **❌ 清除**: 红色边框，轮廓样式 + +## 🔧 技术实现 + +### 架构设计 + +``` +ElementsDisplay.tsx (UI层) + ↓ 发送消息 +content.ts (路由层) + ↓ 调用函数 +element-highlighter.ts (核心逻辑) + ├── locateElementBySelectorType() - 按类型定位 + ├── highlightAllElements() - 批量高亮 + └── clearAllHighlights() - 清除所有 +``` + +### 核心函数 + +#### 1. `locateElementBySelectorType()` +```typescript +function locateElementBySelectorType( + selectors: ElementSelectors, + extractId: string | undefined, + selectorType: SelectorType +): HTMLElement | null +``` +- 根据指定的选择器类型定位元素 +- 支持 9 种选择器类型 +- 返回找到的元素或 null + +#### 2. `highlightAllElements()` +```typescript +export function highlightAllElements( + elementsData: Array<{ selectors: ElementSelectors; extractId?: string }>, + selectorType: SelectorType = 'auto' +): { total: number; success: number; failed: number; unavailable: number } +``` +- 批量高亮多个元素 +- 使用指定的选择器类型 +- 返回详细的统计信息 + +#### 3. `clearAllHighlights()` +```typescript +export function clearAllHighlights(): void +``` +- 停止所有动画 +- 移除所有覆盖层 +- 清理事件监听器 + +### 数据结构 + +#### 覆盖层管理 +```typescript +let allHighlightOverlays: Map = new Map(); +``` + +#### 统计信息 +```typescript +{ + total: 200, // 总元素数 + success: 45, // 成功高亮数 + failed: 35, // 定位失败数 + unavailable: 120 // 无此选择器数 +} +``` + +### 消息通信 + +#### 高亮所有元素 +```typescript +chrome.runtime.sendMessage({ + action: 'highlightAllElements', + elements: [{ selectors, extractId }, ...], + selectorType: 'cssById' +}) +``` + +#### 清除所有高亮 +```typescript +chrome.runtime.sendMessage({ + action: 'clearAllHighlights' +}) +``` + +## 📖 使用场景 + +### 场景 1: 测试 CSS ID 的覆盖率 +``` +1. 选择"CSS by ID (#id)" +2. 点击"高亮所有" +3. 查看统计：成功 45/200，无此选择器 155 + → 说明只有 22.5% 的元素有 ID +``` + +### 场景 2: 对比 XPath 和 CSS 的可靠性 +``` +1. 选择"XPath Short"，点击"高亮所有" + 观察：成功 180/200 + +2. 清除高亮 + +3. 选择"CSS Combined"，点击"高亮所有" + 观察：成功 170/200 + +结论：XPath Short 更可靠（90% vs 85%） +``` + +### 场景 3: 验证提取的数据完整性 +``` +1. 选择"Extract ID"（应该 100% 成功） +2. 点击"高亮所有" +3. 如果有失败，说明提取过程有问题 +``` + +### 场景 4: 测试特定分类的选择器 +``` +1. 切换到"按钮"标签页 +2. 选择"CSS by Class" +3. 点击"🎯 当前分类" +4. 查看按钮元素的 class 选择器覆盖率 +``` + +## 🎯 性能优化 + +### 动画性能 +- 使用 `requestAnimationFrame` 实现 60fps 动画 +- 每个覆盖层独立动画，避免阻塞 + +### 内存管理 +- 使用 `Map` 存储覆盖层，O(1) 查找 +- 自动清理：10 秒后自动移除所有覆盖层 +- 手动清理：提供清除按钮立即释放资源 + +### 事件优化 +- 使用事件委托 +- 防止事件冒泡 (`stopPropagation`) +- 统一的 scroll/resize 监听器 + +## ⚠️ 注意事项 + +### 限制 +1. **大量元素**: 同时高亮 500+ 元素可能影响性能 +2. **动态页面**: SPA 页面动态加载的元素可能定位失败 +3. **iframe**: iframe 内的元素无法高亮 + +### 最佳实践 +1. 优先使用"当前分类"而不是"所有元素" +2. 测试完选择器后及时清除高亮 +3. 对于复杂页面，使用"智能优先"模式 + +## 🔍 调试技巧 + +### 查看控制台日志 +```javascript +// content script 日志 +Highlighted 45/200 elements using cssById +Unavailable: 120, Failed: 35 + +// 单个元素定位日志 +Element found by CSS ID: #submit-button +``` + +### 测试选择器 +1. 打开开发者工具 +2. 使用 `document.querySelector()` 测试选择器 +3. 对比不同选择器的结果 + +## 📊 数据分析 + +### 统计报告示例 +``` +测试页面: https://example.com +总元素数: 200 + +选择器类型覆盖率： +- Extract ID: 100% (200/200) ✅ +- CSS by ID: 22.5% (45/200) +- XPath Short: 90% (180/200) +- CSS Combined: 85% (170/200) +- CSS by Class: 75% (150/200) + +结论： +1. Extract ID 最可靠（提取时添加） +2. XPath Short 覆盖率最高 +3. CSS by ID 覆盖率较低，建议使用其他选择器 +``` + +## 🚀 未来规划 + +### v2.5 可能的增强 +- [ ] 导出高亮统计报告（CSV/JSON） +- [ ] 选择器性能对比（响应时间） +- [ ] 自定义高亮颜色 +- [ ] 批量高亮的动画配置 +- [ ] 选择器自动推荐（根据覆盖率） + +## 📚 相关文档 + +- [README.md](README.md) - 完整项目文档 +- [selector.md](selector.md) - 选择器生成原理 +- [element-highlighter.ts](src/content/element-highlighter.ts) - 核心实现 + +## 🎉 总结 + +v2.4.0 的批量高亮功能为用户提供了强大的选择器测试和验证工具： + +✅ **9种选择器类型** - 全面覆盖各种定位需求 +✅ **实时统计反馈** - 清晰了解选择器可靠性 +✅ **批量操作** - 提升测试效率 +✅ **优雅的UI** - Material-UI 设计规范 + +--- + +**开发者**: AI Assistant +**版本**: v2.4.0 +**日期**: 2025-11-10 + diff --git a/MIGRATION.md b/MIGRATION.md deleted file mode 100644 index 9712b85..0000000 --- a/MIGRATION.md +++ /dev/null @@ -1,217 +0,0 @@ -# 🔄 Migration Guide: Vanilla JS → React + TypeScript - -## ✅ Migration Completed Successfully! - -The SingleFile Lite Chrome extension has been successfully migrated from vanilla JavaScript to a modern tech stack with React 18, TypeScript 5.9, and Material-UI v7. - -## 📦 What Was Done - -### 1. **Project Structure Transformation** -- Created `src/` directory with modular organization -- Split code into `sidepanel/`, `background/`, and `content/` directories -- Organized React components in `src/sidepanel/components/` -- Separated business logic into `src/sidepanel/services/` - -### 2. **Tech Stack Upgrade** -- **UI**: Vanilla HTML/CSS → React 18 + Material-UI v7 -- **Language**: JavaScript → TypeScript 5.9 -- **Build**: None → Webpack 5 -- **Styling**: Inline CSS → Emotion CSS-in-JS -- **Type Safety**: None → Full TypeScript type definitions - -### 3. **Code Migration** - -#### Old Files → New Files: -| Old File | New File | Description | -|----------|----------|-------------| -| `sidepanel.html` | `src/sidepanel/index.html` | HTML template | -| `sidepanel.js` | `src/sidepanel/App.tsx` | Main React component | -| | `src/sidepanel/components/*.tsx` | React UI components | -| `element-extractor.js` | `src/sidepanel/services/elementExtractor.ts` | Element extraction logic | -| `export-utils.js` | `src/sidepanel/services/exportService.ts` | Export functionality | -| `background.js` | `src/background/background.ts` | Background service worker | -| `content.js` | `src/content/content.ts` | Content script | -| `manifest.json` | `src/public/manifest.json` | Extension manifest | - -### 4. **New Files Created** -- `package.json` - Dependencies and npm scripts -- `tsconfig.json` - TypeScript configuration -- `webpack.config.js` - Build configuration -- `src/sidepanel/types/index.ts` - Type definitions -- `.gitignore` - Git ignore rules - -## 🗂️ Old Files in Root Directory - -The following old files remain in the root directory and can be **archived or removed**: - -### Legacy Files (No Longer Used): -- ❌ `background.js` - Replaced by `src/background/background.ts` -- ❌ `content.js` - Replaced by `src/content/content.ts` -- ❌ `element-extractor.js` - Replaced by `src/sidepanel/services/elementExtractor.ts` -- ❌ `export-utils.js` - Replaced by `src/sidepanel/services/exportService.ts` -- ❌ `sidepanel.html` - Replaced by `src/sidepanel/index.html` -- ❌ `sidepanel.js` - Replaced by `src/sidepanel/App.tsx` + components -- ❌ `manifest.json` - Replaced by `src/public/manifest.json` -- ❌ `inline.js` - No longer needed - -### Keep These Files: -- ✅ `icon*.png` - Icon files (copied to dist/ during build) -- ✅ `icon.svg` - Source icon file -- ✅ `icon-generator.html` - Utility file -- ✅ `README.md` - Updated documentation -- ✅ `SingleFile技术原理学习文档.md` - Learning documentation -- ✅ `安装指南.txt` - Installation guide - -## 🚀 How to Use the New System - -### Development Workflow: -```bash -# 1. Install dependencies (one time) -npm install - -# 2. Start development with watch mode -npm run dev - -# 3. Load the dist/ folder in Chrome as unpacked extension - -# 4. Make changes to src/ files -# Webpack auto-rebuilds → Refresh extension in Chrome -``` - -### Production Build: -```bash -npm run build -# Output goes to dist/ folder -``` - -### Type Checking: -```bash -npm run type-check -# Verify TypeScript types without building -``` - -## 🎯 Key Benefits of Migration - -### 1. **Type Safety** -- ✅ Catch errors at compile time -- ✅ Better IDE autocomplete -- ✅ Self-documenting code with type definitions - -### 2. **Modern UI** -- ✅ Material-UI components for consistent design -- ✅ Better user experience -- ✅ Responsive and accessible - -### 3. **Better Code Organization** -- ✅ Modular structure -- ✅ Separation of concerns -- ✅ Easier to maintain and extend - -### 4. **Developer Experience** -- ✅ Hot reload during development -- ✅ Source maps for debugging -- ✅ Build optimization - -## 📊 File Comparison - -### Before (Vanilla JS): -``` -project/ -├── background.js (3.5 KB) -├── content.js (16 KB) -├── element-extractor.js (6 KB) -├── export-utils.js (11 KB) -├── sidepanel.html (10 KB) -├── sidepanel.js (32 KB) -└── manifest.json (1 KB) -Total: ~80 KB unorganized code -``` - -### After (React + TypeScript): -``` -src/ -├── background/background.ts (3 KB) -├── content/content.ts (17 KB) -├── sidepanel/ -│ ├── App.tsx (4 KB) -│ ├── components/ -│ │ ├── SavePageButton.tsx (6 KB) -│ │ ├── ExtractButton.tsx (7 KB) -│ │ ├── ProgressBar.tsx (1 KB) -│ │ └── StatusMessage.tsx (1 KB) -│ ├── services/ -│ │ ├── pageExtractor.ts (13 KB) -│ │ ├── elementExtractor.ts (8 KB) -│ │ └── exportService.ts (14 KB) -│ └── types/index.ts (6 KB) -Total: ~80 KB well-organized, type-safe code -``` - -## 🔄 What Changed in Functionality? - -### Nothing! ✨ -All features work exactly the same: -- ✅ Save webpage as single HTML file -- ✅ Extract interactive elements -- ✅ Export to HTML/Excel/CSV/JSON -- ✅ Side panel UI -- ✅ Progress tracking -- ✅ CORS proxy fallback - -### What's Better: -- 🎨 More polished UI with Material-UI -- 🐛 Better error handling with TypeScript -- 🚀 Faster development with hot reload -- 📦 Professional build system -- 🔧 Easier to maintain and extend - -## 🧹 Clean Up Recommendation - -You can safely move these old files to an `_archive/` folder: -```bash -mkdir _archive -move background.js _archive/ -move content.js _archive/ -move element-extractor.js _archive/ -move export-utils.js _archive/ -move sidepanel.html _archive/ -move sidepanel.js _archive/ -move manifest.json _archive/ -move inline.js _archive/ -``` - -Or delete them if you're confident in the migration: -```bash -# ⚠️ Only do this if you're sure! -rm background.js content.js element-extractor.js export-utils.js -rm sidepanel.html sidepanel.js manifest.json inline.js -``` - -## 📚 Next Steps - -1. **Test the extension**: Load `dist/` folder in Chrome and test all features -2. **Familiarize yourself** with the new structure in `src/` -3. **Read the updated** `README.md` for development instructions -4. **Start developing** with `npm run dev` -5. **Archive old files** when ready - -## 🆘 Troubleshooting - -### Extension won't load? -→ Make sure you loaded the **`dist/`** folder, not the root directory - -### Build errors? -→ Run `npm install` first, then `npm run build` - -### Changes not showing? -→ Rebuild with `npm run dev`, then refresh extension in Chrome - -### Type errors? -→ Run `npm run type-check` to see detailed TypeScript errors - -## 🎉 Conclusion - -The migration is complete! You now have a modern, maintainable, type-safe Chrome extension built with industry-standard tools. The old vanilla JavaScript files can be archived or removed. - -Happy coding! 🚀 - diff --git a/MIGRATION_SUMMARY.md b/MIGRATION_SUMMARY.md deleted file mode 100644 index 820e45a..0000000 --- a/MIGRATION_SUMMARY.md +++ /dev/null @@ -1,264 +0,0 @@ -# ✅ Migration Complete - SingleFile Lite - -## 🎉 Success! All Tasks Completed - -The SingleFile Lite Chrome extension has been successfully migrated from vanilla JavaScript to a modern tech stack with **React 18 + TypeScript 5.9 + Material-UI v7 + Webpack 5**. - ---- - -## 📋 Completed Tasks - -### ✅ Phase 1: Project Setup -- Created `package.json` with all dependencies -- Configured `tsconfig.json` for React + TypeScript -- Set up `webpack.config.js` with multiple entry points -- Added `.gitignore` for version control - -### ✅ Phase 2: Directory Structure -- Created `src/` with modular organization -- Set up `sidepanel/`, `background/`, `content/` directories -- Organized React components in `components/` -- Separated business logic in `services/` - -### ✅ Phase 3: Type Definitions -- Defined all TypeScript interfaces in `src/sidepanel/types/index.ts` -- Type-safe message passing between extension parts -- Proper typing for Chrome Extension APIs - -### ✅ Phase 4: Services Migration -- Ported `element-extractor.js` → `elementExtractor.ts` -- Ported `export-utils.js` → `exportService.ts` -- Created `pageExtractor.ts` from sidepanel.js logic -- All with full TypeScript type safety - -### ✅ Phase 5: Background & Content Scripts -- Migrated `background.js` → `background.ts` -- Migrated `content.js` → `content.ts` -- Maintained vanilla implementation (no React) -- Added proper type definitions - -### ✅ Phase 6: React Components -- Created `App.tsx` with Material-UI theme -- Built `SavePageButton.tsx` for page saving -- Built `ExtractButton.tsx` for element extraction -- Created `ProgressBar.tsx` and `StatusMessage.tsx` -- All using MUI components and TypeScript - -### ✅ Phase 7: Build & Test -- Successfully built with Webpack 5 -- Output to `dist/` folder -- All files generated correctly -- No TypeScript errors - -### ✅ Phase 8: Documentation -- Updated `README.md` with comprehensive instructions -- Created `MIGRATION.md` guide -- Created `.gitignore` for best practices - ---- - -## 📦 Build Output (dist/ folder) - -``` -dist/ -├── sidepanel.html (471 bytes) -├── sidepanel.js (326 KB - React + MUI bundle) -├── background.js (1.4 KB) -├── content.js (7.6 KB) -├── manifest.json (896 bytes) -├── icon16.png (689 bytes) -├── icon48.png (2.9 KB) -└── icon128.png (13.3 KB) -``` - -**Total build size**: ~350 KB (optimized for production) - ---- - -## 🚀 How to Use - -### First Time Setup: -```bash -cd d:\backend_learning -npm install # Already done ✅ -npm run build # Already done ✅ -``` - -### Load Extension: -1. Open Chrome: `chrome://extensions/` -2. Enable "Developer mode" -3. Click "Load unpacked" -4. Select the **`dist/`** folder -5. Done! 🎉 - -### Development: -```bash -npm run dev # Watch mode - auto rebuild on changes -# Then refresh extension in Chrome after changes -``` - ---- - -## 📊 Technology Stack - -| Component | Technology | Version | -|-----------|-----------|---------| -| UI Framework | React | 18.2.0 | -| Language | TypeScript | 5.9 | -| Component Library | Material-UI | 7.3.4 | -| CSS-in-JS | Emotion | 11.11.4 | -| Build Tool | Webpack | 5.94.0 | -| HTTP Client | Axios | 1.7.7 | -| Chrome APIs | @types/chrome | 0.0.270 | - ---- - -## 🎨 Features Preserved - -All original features work exactly as before: - -✅ **Save Webpage** -- Single HTML file with inlined resources -- Image/CSS/icon handling -- Lazy loading support -- Shadow DOM processing -- Tracker removal - -✅ **Extract Elements** -- Buttons, links, forms, inputs -- 4 export formats (HTML/Excel/CSV/JSON) -- Complete attribute extraction -- Custom component detection - -✅ **Side Panel UI** -- Opens on icon click -- Material-UI design -- Progress tracking -- Status messages - ---- - -## 🔧 Project Structure - -``` -src/ -├── sidepanel/ # React App -│ ├── App.tsx # Main component -│ ├── index.tsx # React entry -│ ├── index.html # HTML template -│ ├── components/ # React components -│ │ ├── SavePageButton.tsx -│ │ ├── ExtractButton.tsx -│ │ ├── ProgressBar.tsx -│ │ └── StatusMessage.tsx -│ ├── services/ # Business logic -│ │ ├── pageExtractor.ts -│ │ ├── elementExtractor.ts -│ │ └── exportService.ts -│ └── types/ # TypeScript types -│ └── index.ts -├── background/ # Background script -│ └── background.ts -├── content/ # Content script -│ └── content.ts -└── public/ # Static assets - ├── manifest.json - └── icons/ - -dist/ # Build output (load this!) -node_modules/ # Dependencies -package.json # npm config -tsconfig.json # TypeScript config -webpack.config.js # Webpack config -README.md # Updated docs -MIGRATION.md # Migration guide -.gitignore # Git ignore rules -``` - ---- - -## 📝 NPM Scripts - -| Command | Description | -|---------|-------------| -| `npm run dev` | Development build with watch mode | -| `npm run build` | Production build (optimized) | -| `npm run type-check` | TypeScript type checking only | - ---- - -## 🧹 Cleanup Recommendation - -### Old Files (Can Be Removed): -These files in the root directory are no longer used: -- `background.js` -- `content.js` -- `element-extractor.js` -- `export-utils.js` -- `sidepanel.html` -- `sidepanel.js` -- `manifest.json` -- `inline.js` - -**Recommendation**: Move to `_archive/` folder or delete - -### Files to Keep: -- `icon*.png` - Still used (copied during build) -- `icon.svg` - Source file -- `icon-generator.html` - Utility -- `README.md` - Updated documentation -- Documentation files - ---- - -## ⚠️ Important Notes - -1. **Load the `dist/` folder** in Chrome, NOT the root directory -2. **Rebuild after changes**: Run `npm run dev` or `npm run build` -3. **Refresh extension** in Chrome after rebuilding -4. **Old files** in root are legacy and can be archived - ---- - -## 🎯 Next Steps - -1. ✅ Load the extension from `dist/` folder -2. ✅ Test all features (save page, extract elements) -3. ✅ Verify side panel opens correctly -4. ✅ Check that exports work (HTML/Excel/CSV/JSON) -5. ⚠️ Archive old files when ready -6. 🚀 Start developing with the new stack! - ---- - -## 📚 Documentation - -- **README.md**: Complete usage and development guide -- **MIGRATION.md**: Detailed migration information -- **This file**: Quick summary - ---- - -## 🎉 Conclusion - -The migration was **100% successful**! - -- ✅ All features work -- ✅ Modern tech stack -- ✅ Type-safe code -- ✅ Better UI/UX -- ✅ Easy to maintain -- ✅ Production ready - -**The extension is ready to use and develop!** - ---- - -**Date**: 2025-11-05 -**Version**: 2.0.0 -**Status**: ✅ Migration Complete -**Build**: ✅ Successful -**Ready**: ✅ Production Ready - -🚀 Happy Coding! - diff --git a/QUICK_START_BUTTON_FIX.md b/QUICK_START_BUTTON_FIX.md new file mode 100644 index 0000000..198794b --- /dev/null +++ b/QUICK_START_BUTTON_FIX.md @@ -0,0 +1,119 @@ +# ⚡ 按钮选择器修复 - 快速开始 + +## 🎯 修复内容 + +已成功更新按钮提取选择器，现在可以识别和提取**包含 "btn" 的所有 `` 标签**。 + +## ✅ 修复验证 + +✔️ **构建状态**: ✅ 成功 +✔️ **文件修改**: `src/content/element-extractor.ts`（第 37 行） +✔️ **代码检查**: 无错误 +✔️ **新增文档**: `src/content/BUTTON_SELECTOR_UPDATE.md` + +## 🚀 立即使用 + +### 步骤 1: 刷新扩展 +``` +1. 打开 Chrome 扩展页面：chrome://extensions/ +2. 找到 "SingleFile Lite" 扩展 +3. 点击刷新按钮 ↻ +``` + +### 步骤 2: 测试效果 +``` +1. 打开含有分页按钮的页面（如数据表格页面） +2. 点击扩展图标打开侧边栏 +3. 点击"提取可交互元素" +4. 在"按钮"标签页中查看，应该能看到： + - 首页按钮 (home) + - 上一页按钮 (prev) + - 下一页按钮 (next) + - 末页按钮 (end) +``` + +### 步骤 3: 验证高亮功能 +``` +1. 选择选择器类型："🎯 智能优先" 或 "🟢 Extract ID" +2. 点击"✨ 高亮所有"按钮 +3. 所有分页按钮应该显示高亮覆盖层 +4. 查看统计信息验证提取成功 +``` + +## 📊 修改内容一览 + +### 修改前 +``` +页面类选择器包含： +✅ button, [role="button"], input[type="button"], ... +✅ a.btn, a.button, a.ant-btn, a.el-button, ... +✅ a[class*="bh-btn"], a[class*="-button"], a[class*="btn-"] +❌ a[class*="btn"] ← 缺少这个！ +``` + +### 修改后 +``` +页面类选择器包含： +✅ button, [role="button"], input[type="button"], ... +✅ a.btn, a.button, a.ant-btn, a.el-button, ... +✅ a[class*="bh-btn"], a[class*="-button"], a[class*="btn-"] +✅ a[class*="btn"] ← 已添加！ +``` + +## 📈 预期改进 + +| 指标 | 改进 | +|------|------| +| 分页按钮提取 | ❌ 无 → ✅ 有 | +| 总按钮提取数 | +2-5% | +| 选择器覆盖率 | +2-5% | +| 用户体验 | 获得更完整的元素数据 | + +## 🔍 技术细节 + +**新增选择器**: `a[class*="btn"]` + +这个 CSS 属性选择器可以匹配： +- `class="bh-pager-btn"` ✅ +- `class="btn-primary"` ✅ +- `class="button btn-sm"` ✅ +- `class="any-btn-anything"` ✅ + +任何 `` 标签的 class 属性中包含 `"btn"` 字符串的元素。 + +## 🐛 已知问题 + +**无已知问题** + +如果发现新问题，请检查： +1. 是否已刷新扩展？ +2. 是否是 SPA 页面？（页面切换可能需要重新提取） +3. 是否正确加载了最新构建文件？ + +## 💡 常见问题 + +### Q: 为什么按钮数量增加了？ +**A**: 因为现在可以识别之前遗漏的自定义按钮。这是预期行为。 + +### Q: 会不会提取错误的链接？ +**A**: 极少数情况。浏览器原生选择器会精确匹配包含 "btn" 的类名，误捕获率非常低。 + +### Q: 如何撤销这个修改？ +**A**: 将第 37 行的 `a[class*="btn"]` 删除即可恢复原状。 + +## 📚 更多信息 + +- 📖 详细文档: `src/content/BUTTON_SELECTOR_UPDATE.md` +- 🔧 源代码: `src/content/element-extractor.ts` +- 📋 项目文档: `README.md` + +## ✨ 总结 + +这个修复使扩展能够**识别更多类型的按钮元素**，特别是自定义框架提供的按钮。修改非常小（仅添加一个选择器），但能显著提升提取的完整性和用户体验。 + +--- + +**修改日期**: 2025-11-11 +**版本**: v2.6.2 +**状态**: ✅ 已部署 + diff --git a/README.md b/README.md index d196853..393fce5 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ **一键保存网页 + 提取可交互元素 | 侧边栏UI，更大空间，更好体验** -## 🚀 两大核心功能 +## 🚀 三大核心功能 ### 1. 💾 保存网页为单文件 将网页保存为完整的HTML文件，所有资源内联（图片、CSS、图标） @@ -12,13 +12,19 @@ - 移除追踪代码（Google Analytics、百度统计等） - CORS代理支持跨域资源 -### 2. 📋 提取可交互元素 ⭐ 新功能 +### 2. 📋 提取可交互元素 一键提取页面所有可交互元素，在侧边栏中分类展示，支持元素定位和选择器生成 +### 3. 🎬 Viewer 模式 - 录制与回放 ⭐ 新功能 +录制用户在页面上的所有操作，支持导出/导入会话，并可自动回放重现用户行为 + **提取内容**： -- **元素类型**：按钮、链接、表单、输入框、下拉框、文本域、自定义组件 -- **完整属性**：id、class、name、type、placeholder、验证规则等 +- **元素类型**：按钮、链接、表单、输入框、下拉框、文本域、复选框、单选框、开关、滑块、**数据表格**、自定义组件 +- **框架支持**：原生HTML + jQWidgets 全面支持（ListBox、Grid、xtype、.jqx-* 类、ARIA roles）+ EMAP 数据表格 +- **完整属性**：id、class、name、type、placeholder、checked、value、范围等 - **自定义属性**：data-*、aria-label、onclick 事件等 +- **表格识别**：自动识别 jQWidgets、EMAP、AG-Grid 等数据表格组件，提取行列结构、分页信息、排序筛选功能 +- **智能清理**：SPA 页面切换时自动清理旧标记，避免元素提取冲突 **智能定位**： - **多重选择器**：为每个元素生成 XPath、CSS（ID/Class/Attribute/nth-child）等多种选择器 @@ -124,6 +130,8 @@ npm run type-check **交互功能**： - 📊 **分类展示** - 按钮、链接、表单等分标签页显示，一目了然 - 🎯 **元素定位** - 点击任意元素，页面自动滚动到该元素并高亮显示 +- ✨ **批量高亮** - 一键高亮所有元素或当前分类，支持选择器类型选择 ⭐ 新增 +- 🎛️ **选择器测试** - 9种选择器类型可选，测试不同选择器的可靠性 - 📋 **选择器列表** - 展开元素查看所有可用选择器（XPath、CSS等） - 📝 **复制选择器** - 一键复制选择器，方便用于自动化测试或爬虫开发 - 🟢 **可靠性指示** - 不同颜色标识选择器的稳定性和唯一性 @@ -134,11 +142,73 @@ npm run type-check 2. 点击扩展图标打开侧边栏 3. 点击"提取可交互元素"按钮 4. 查看分类统计和元素列表 + +单个元素交互： 5. 点击表格中任一行 → 页面滚动 + 元素高亮 6. 点击展开图标 ▼ → 查看该元素的所有选择器 7. 点击复制图标 📋 → 选择器已复制到剪贴板 + +批量高亮（新功能）： +8. 选择"选择器类型"下拉框 → 选择要测试的选择器类型 +9. 点击"✨ 高亮所有"→ 页面上所有提取元素同时高亮 + 或点击"🎯 当前分类"→ 只高亮当前标签页的元素 +10. 查看高亮结果统计 → 成功数/失败数/不可用数 +11. 点击"❌ 清除"→ 手动清除所有高亮（或等待10秒自动清除） +``` + +### 📹 Viewer 模式使用 + +**切换到 Viewer 模式**： +``` +侧边栏顶部点击 "🎬 Viewer Mode" 标签页 ``` +**录制用户操作**： +``` +1. 点击 "🔴 Start Recording" 开始录制 +2. 在页面上执行各种操作（点击、滚动、输入等） +3. 实时显示录制时长和操作数 +4. 点击 "⏸️ Pause" 暂停录制 / "▶️ Resume" 继续录制 +5. 点击 "⏹️ Stop" 停止并保存录制 +``` + +**导出与导入**： +``` +1. 点击 "💾 Export" 导出录制为 JSON 文件 +2. 点击 "📂 Import" 导入已有的录制文件 +3. 点击 "📊 View Stats" 查看操作统计信息 +4. 点击 "🗑️ Delete" 清除当前会话 +``` + +**回放录制**： +``` +1. 确保已加载录制会话（录制后或导入后） +2. 选择播放速度（0.5x / 1x / 2x / 5x） +3. 点击 "▶️ Play" 开始自动回放 +4. 观察页面自动重现录制的操作（带高亮提示） +5. 使用 "⏩ +5s" / "⏪ -5s" 快进/快退 +6. 点击 "⏸️ Pause" 暂停 / "⏹️ Stop" 停止回放 +``` + +**时间轴与操作列表**： +``` +1. 时间轴显示所有操作点，不同颜色代表不同操作类型 +2. 点击时间轴上的点 → 跳转到该操作并高亮元素 +3. 操作列表显示详细信息（时间、类型、目标元素） +4. 点击列表中的操作 → 在页面中定位并高亮该元素 +5. 展开操作查看完整的选择器信息 +``` + +**支持的操作类型**： +- 🖱️ **鼠标操作**：点击、双击、按下、释放、移动、进入、离开 +- 📜 **滚动操作**：页面滚动、元素内滚动 +- ⌨️ **输入操作**：文本输入、内容变化（敏感字段自动隐藏） +- 🔤 **键盘操作**：按键按下、释放（含修饰键） +- 🎯 **焦点操作**：聚焦、失焦 +- 🔀 **拖拽操作**：拖动开始、结束、放置 +- 📤 **表单操作**：表单提交 +- 🔗 **导航操作**：URL 变化（SPA 路由跳转） + ## 📂 项目结构 ``` @@ -147,36 +217,39 @@ d:\backend_learning\ │ ├── 📁 sidepanel/ # React侧边栏应用 │ │ ├── App.tsx # 主React组件 │ │ ├── index.tsx # React入口 -│ │ ├── index.html # HTML模板 -│ │ ├── 📁 components/ # React组件 -│ │ │ ├── SavePageButton.tsx # 保存页面按钮 -│ │ │ ├── ExtractButton.tsx # 提取元素按钮 -│ │ │ ├── ElementsDisplay.tsx # 元素展示组件（分类表格+选择器） -│ │ │ ├── ProgressBar.tsx # 进度条 -│ │ │ └── StatusMessage.tsx # 状态消息 -│ │ ├── 📁 services/ # 业务逻辑 -│ │ │ ├── pageExtractor.ts # 页面保存服务 -│ │ │ └── elementExtractor.ts # 元素提取服务（已废弃，逻辑在content.ts） +│ │ ├── 📁 components/ # React组件（按钮、展示、分类表格） +│ │ ├── 📁 services/ # 业务逻辑（页面保存服务） │ │ └── 📁 types/ # TypeScript类型定义 -│ │ └── index.ts │ ├── 📁 background/ # 后台服务 -│ │ └── background.ts -│ ├── 📁 content/ # 内容脚本 -│ │ └── content.ts -│ └── 📁 public/ # 静态资源 -│ ├── manifest.json # 扩展清单 -│ └── icons/ +│ │ └── background.ts # Service Worker（消息转发、CORS代理） +│ ├── 📁 content/ # 内容脚本（模块化架构） +│ │ ├── content.ts # 入口：消息路由 +│ │ ├── page-extractor.ts # 页面保存逻辑 +│ │ ├── element-extractor.ts # 元素提取逻辑 +│ │ ├── 📁 element-highlighter/ # 元素高亮模块（5个子模块） +│ │ │ ├── index.ts # 主入口（单个/批量高亮） +│ │ │ ├── element-locator.ts # 7级回退定位 +│ │ │ ├── overlay-manager.ts # 覆盖层创建与动画 +│ │ │ ├── info-panel.ts # 信息面板（选择器/HTML展示） +│ │ │ └── utils.ts # 辅助函数（复制、转义等） +│ │ ├── 📁 viewer-recorder/ # Viewer 录制模块（5个子模块）⭐ 新增 +│ │ │ ├── index.ts # 录制器主入口 +│ │ │ ├── event-listeners.ts # 事件监听管理 +│ │ │ ├── action-serializer.ts# 操作序列化 +│ │ │ ├── scroll-tracker.ts # 滚动追踪 +│ │ │ └── storage-manager.ts # 数据存储 +│ │ ├── 📁 viewer-player/ # Viewer 回放模块（4个子模块）⭐ 新增 +│ │ │ ├── index.ts # 回放器主入口 +│ │ │ ├── action-executor.ts # 操作执行 +│ │ │ ├── timeline-controller.ts # 时间轴控制 +│ │ │ └── playback-highlighter.ts # 回放高亮 +│ │ ├── 📁 selectors/ # 选择器生成（XPath、CSS） +│ │ └── 📁 utils/ # 工具函数（资源处理、追踪器移除） +│ └── 📁 public/ # 静态资源（manifest、icons） ├── 📁 dist/ # Webpack构建输出（加载此目录） -│ ├── sidepanel.html -│ ├── sidepanel.js -│ ├── background.js -│ ├── content.js -│ ├── manifest.json -│ └── *.png # 图标文件 ├── 📄 package.json # 依赖配置 ├── 📄 tsconfig.json # TypeScript配置 -├── 📄 webpack.config.js # Webpack配置 -└── 📄 README.md # 本文件 +└── 📄 webpack.config.js # Webpack配置 ``` ## 🔧 技术架构 @@ -185,74 +258,173 @@ d:\backend_learning\ ``` 用户点击图标 → background.ts 打开侧边栏 ↓ - React App (sidepanel) - ├── 💾 保存页面 - │ └── 注入 pageExtractor → 内联资源 → 下载HTML - └── 📋 提取元素 - └── content script 提取元素 + 生成选择器 - ↓ - ElementsDisplay 组件展示 - ↓ - 点击元素 → background → content script → 高亮显示 + React App (sidepanel) - 标签页模式 + ├── 💾 Page Saver Tab + │ ├── 保存页面 → content/page-extractor → 内联资源 → 下载HTML + │ └── 提取元素 → content/element-extractor → 生成选择器 + │ ↓ + │ ElementsDisplay 展示 + │ ↓ + │ 点击元素 → content/element-highlighter → 高亮 + │ + └── 🎬 Viewer Mode Tab ⭐ 新增 + ├── 录制模式 → content/viewer-recorder + │ ├── event-listeners → 捕获所有用户操作 + │ ├── action-serializer → 序列化为 UserAction + │ ├── scroll-tracker → 追踪滚动事件 + │ └── storage-manager → 批量存储/导出 JSON + │ + └── 回放模式 → content/viewer-player + ├── action-executor → 执行操作（dispatchEvent） + ├── timeline-controller → 时间轴控制（播放/暂停/跳转） + └── playback-highlighter → 回放高亮（彩色覆盖层） ``` -### 关键技术点 +### 模块化架构（content scripts） +- **content.ts** - 消息路由入口 +- **page-extractor.ts** - 页面保存（图片、CSS内联） +- **element-extractor.ts** - 元素提取（按钮、链接、表单、数据表格等12类） +- **element-highlighter/** - 高亮模块（5个子模块） + - `index.ts` - 主入口，单个/批量高亮逻辑 + - `element-locator.ts` - 7级回退定位策略 + - `overlay-manager.ts` - 覆盖层创建与60fps动画 + - `info-panel.ts` - 信息面板（ℹ️按钮、选择器展示） + - `utils.ts` - 辅助函数（HTML转义、复制、Toast） +- **viewer-recorder/** - 录制模块（5个子模块）⭐ 新增 + - `index.ts` - 录制器主入口（生命周期管理） + - `event-listeners.ts` - 全类型事件监听（捕获阶段） + - `action-serializer.ts` - 事件序列化（复用 selector-generator） + - `scroll-tracker.ts` - 滚动追踪（节流 + MutationObserver） + - `storage-manager.ts` - 批量存储（chrome.storage.local） +- **viewer-player/** - 回放模块（4个子模块）⭐ 新增 + - `index.ts` - 回放器主入口（播放控制） + - `action-executor.ts` - 操作执行（dispatchEvent） + - `timeline-controller.ts` - 时间轴控制（播放/暂停/跳转/速度） + - `playback-highlighter.ts` - 回放高亮（彩色覆盖层） +- **selectors/** - XPath、CSS选择器生成 +- **utils/** - 资源转换、追踪器移除 -#### 1. React + Material-UI -- 使用MUI组件构建现代化UI -- ThemeProvider提供紫色渐变主题 -- TypeScript类型安全保证 +### 关键技术点 -#### 2. Webpack构建 -- 多入口点配置（sidepanel、background、content） -- ts-loader编译TypeScript -- 生产环境代码压缩优化 +#### 1. 数据表格提取（v2.6） +**智能识别多种表格框架**： +- jQWidgets Grid（`.jqx-grid[role="grid"]`） +- EMAP 数据表格（`[emap-role="datatable"]`） +- AG-Grid（`.ag-root`） +- 原生表格（`table.datatable`） + +**提取完整表格信息**： +- 结构信息：行数、列数、列名 +- 分页信息：当前页/总页数/总记录数 +- 功能检测：是否支持排序、筛选、分页 +- 元数据：EMAP action/pagePath/appName +- 数据预览：前3行数据样本 + +#### 2. 模块化架构（v2.2） +- **单一职责**：每个模块独立功能（84-240行） +- **依赖分层**：utils → selectors → extractors → routing +- **易于维护**：bug隔离、独立测试、清晰文档 #### 3. Chrome Extension V3 -- Service Worker后台脚本 -- Side Panel API侧边栏 -- Scripting API动态注入 -- 类型安全的消息通信 - -#### 4. 脚本注入策略 -- 优先尝试content script通信 -- 失败则使用executeScript动态注入 -- 函数序列化注入（不能使用外部依赖） - -#### 5. 资源内联技术 -- Fetch + FileReader转Base64 -- CORS代理fallback（通过background） -- 超时控制（5秒） -- 懒加载属性识别（10+种） - -#### 6. 元素定位与选择器生成 -- **XPath生成**：完整路径 + 优化短路径（基于ID） -- **CSS选择器**：ID、Class、Attribute、nth-child、组合选择器 -- **临时标记**：`data-extract-id` 属性确保精确定位 -- **7级回退策略**：从最可靠到最不可靠依次尝试 - ``` - 1️⃣ data-extract-id（临时标记）→ 最可靠 🟢 - 2️⃣ CSS by ID（#id）→ 高可靠 🟢 - 3️⃣ XPath（short/full）→ 中等可靠 🟡 - 4️⃣ CSS Combined（标签+类+属性）→ 中等可靠 🟡 - 5️⃣ CSS by Class（.class）→ 中等可靠 🟡 - 6️⃣ CSS by Attribute（[name=...]）→ 中等可靠 🟡 - 7️⃣ CSS nth-child（:nth-child(n)）→ 低可靠 🔴 - ``` -- **动态高亮**：CSS动画 + 自动清理（5秒超时） - -## 📊 架构对比 - -| 层级 | 之前（Vanilla JS） | 现在（React + TS） | -|-----|-------------------|-------------------| -| **UI框架** | 原生HTML/CSS | React 18 + MUI v7 | -| **语言** | JavaScript | TypeScript 5.9 | -| **样式** | 内联CSS | Emotion CSS-in-JS | -| **构建** | 无 | Webpack 5 | -| **类型检查** | ❌ | ✅ 完整类型定义 | -| **组件化** | ❌ | ✅ React组件 | -| **开发体验** | 手动刷新 | 热重载（需刷新扩展）| -| **代码组织** | 单文件 | 模块化目录结构 | +- Service Worker后台脚本（消息转发、CORS代理） +- Side Panel API（更大操作空间） +- 类型安全的消息通信（TypeScript） + +#### 4. 资源内联技术 +- Fetch + FileReader → Base64 +- CORS代理fallback（background权限） +- 懒加载识别（10+属性）、超时控制（5秒） + +#### 5. 智能元素高亮 +**7级回退定位**： +``` +1️⃣ data-extract-id → 最可靠 🟢 +2️⃣ CSS #id → 高可靠 🟢 +3️⃣ XPath → 中等 🟡 +4️⃣ CSS组合 → 中等 🟡 +5️⃣ CSS class/attr → 中等 🟡 +6️⃣ CSS nth-child → 低可靠 🔴 +``` + +**批量高亮功能**（v2.4）： +- 支持9种选择器类型选择（智能优先/Extract ID/CSS/XPath） +- 同时高亮多个元素，独立动画 +- 实时统计反馈（成功/失败/不可用） +- 测试选择器可靠性和覆盖率 + +**覆盖层高亮方案**（v2.3）： +- 固定定位覆盖层（`position: fixed` + `z-index: 2147483647`） +- requestAnimationFrame 脉冲动画（60fps 平滑效果） +- 实时跟随（scroll/resize 事件监听） +- 不受元素遮挡/Shadow DOM 影响 +- 适用于任何页面（包括 Google 等复杂页面） + +#### 6. Viewer 模式 - 录制与回放（v3.0）⭐ 新增 +**录制技术**： +- 事件捕获：使用捕获阶段（`capture: true`）监听所有用户操作 +- 事件节流：mousemove 200ms、scroll 100ms 节流优化性能 +- History API 拦截：捕获 SPA 应用的 pushState/popState 导航 +- 敏感字段过滤：自动隐藏密码、信用卡等敏感输入 +- 批量存储：每 50 个操作或每 5 秒批量写入 chrome.storage.local + +**回放技术**： +- 操作执行：使用 dispatchEvent() 触发原生事件 +- 时间轴控制：按相对时间戳调度操作，支持播放/暂停/跳转 +- 速度控制：支持 0.5x、1x、2x、5x 播放速度 +- 智能等待：MutationObserver 检测 DOM 变化后再执行下一步 +- 彩色高亮：不同操作类型使用不同颜色高亮（蓝=点击、绿=滚动、橙=输入） + +**数据结构**： +```typescript +interface UserAction { + id: string; // 唯一标识 + timestamp: number; // 相对时间戳（ms） + type: UserActionType; // 操作类型（20+种） + target?: { selectors, text, attributes }; // 目标元素 + position?: { x, y, clientX, clientY }; // 鼠标位置 + scroll?: { x, y, targetSelector }; // 滚动信息 + input?: { value, isSensitive }; // 输入内容 + keyboard?: { key, code, modifiers }; // 键盘按键 + navigation?: { from, to, type }; // 导航信息 + viewport: { width, height }; // 视口大小 +} +``` + +**UI 组件**： +- RecordingControl - 录制控制（开始/暂停/停止） +- RecordingTimeline - 可视化时间轴（彩色操作点） +- ActionsList - 操作列表（可搜索、可展开） +- PlaybackControl - 回放控制（播放/快进/速度） +- SessionManager - 会话管理（导出/导入/统计） + +#### 7. React + Material-UI +- MUI组件库 + 紫色渐变主题 +- TypeScript完整类型定义 +- Webpack多入口构建 +- 标签页模式（Page Saver + Viewer Mode） + +## 📊 架构演进 + +| 版本 | 架构特点 | 核心功能 | +|-----|---------|---------| +| **v1.x** | Vanilla JS | 基础页面保存 | +| **v2.0-2.2** | React + TS + 模块化 | 元素提取 + CSS class 高亮 | +| **v2.3** | 覆盖层优化 | 固定定位覆盖层高亮（通用兼容）| +| **v2.4** | 批量高亮 | 多元素同时高亮 + 9种选择器类型 | +| **v2.6** | 数据表格支持 | jQWidgets/EMAP/AG-Grid 表格提取 | +| **v3.0** ⭐ | **Viewer 模式** | **用户操作录制与自动回放** | + +**v2.6.1 SPA 优化收益**： +- ✅ 支持 jQWidgets ListBox 复选框（.jqx-checkbox-default） +- ✅ SPA 页面切换自动清理旧标记 +- ✅ 智能标签关联（chkbox + span 结构） +- ✅ 增强可见性检测（computed style） + +**v2.3 覆盖层收益**： +- ✅ 适配复杂页面（Google、YouTube 等） +- ✅ 不受元素遮挡/Shadow DOM 影响 +- ✅ 立即高亮（无需滚动触发） +- ✅ 平滑动画（60fps requestAnimationFrame） ## ⚠️ 注意事项 @@ -287,14 +459,12 @@ d:\backend_learning\ - 遵循Material Design设计规范 - 保持紫色渐变主题一致性 -### 类型定义 +### 类型定义（`types/index.ts`） -所有主要数据结构都在 `src/sidepanel/types/index.ts` 中定义： -- `SaveOptions` - 保存选项 -- `ElementData` - 提取的元素数据 -- `ElementSelectors` - 元素选择器集合（XPath、CSS等） +- `SaveOptions` / `SaveResult` - 保存配置与结果 +- `ElementData` / `ElementSelectors` - 元素数据与选择器 - `ChromeMessage` - 消息类型（类型安全通信） -- 各种元素类型：`ButtonElement`、`LinkElement`、`FormElement` 等 +- 7种元素类型：Button / Link / Form / Input / Select / Textarea / Custom ### 调试技巧 @@ -307,11 +477,18 @@ d:\backend_learning\ | 版本 | 日期 | 主要更新 | |-----|------|---------| -| **v2.1.0** | 2025-11-05 | 🎯 **元素定位功能**：
✨ 智能选择器生成（XPath + 多种CSS选择器）
🎨 点击元素自动滚动并高亮
📋 可展开查看所有选择器
📝 一键复制选择器到剪贴板
🟢 选择器可靠性标识
🔧 7级回退定位策略 | -| **v2.0.0** | 2025-11-05 | 🎉 **重大重构**：迁移到 React + TypeScript + MUI
✨ 现代化技术栈
🔧 Webpack构建系统
📦 完整类型定义
🎨 Material-UI组件库 | -| v1.2.0 | 2025-11-05 | 🎨 侧边栏UI、更大操作空间、更好体验 | -| v1.1.0 | 2025-11-05 | ✨ 提取元素功能（已重构） | -| v1.0.0 | 2025-11-04 | 🚀 基础保存功能、资源内联、追踪器移除 | +| **v3.0.0** ⭐ | 2025-11-11 | 🎬 **Viewer 模式 - 录制与回放**：
✨ 完整的用户操作录制（20+种事件类型）
🎯 自动回放功能（dispatchEvent 模拟操作）
📊 可视化时间轴（彩色操作点）
💾 导出/导入 JSON 会话文件
⚡ 智能优化（事件节流、批量存储、敏感信息过滤）
🎨 彩色高亮回放（不同操作类型不同颜色）
🎛️ 回放控制（播放/暂停/跳转/速度调节）| +| **v2.6.1** | 2025-11-11 | 🔧 **SPA 优化 + ListBox 支持**：
✨ 支持 jQWidgets ListBox 复选框提取
🧹 自动清理旧 Extract ID（解决 SPA 页面切换冲突）
👁️ 增强可见性检测（过滤 visibility:hidden）
🏷️ 智能标签关联（支持复杂 DOM 结构）| +| **v2.6.0** | 2025-11-11 | 📊 **数据表格支持**：新增数据表格提取功能，支持 jQWidgets/EMAP/AG-Grid | +| **v2.5.0** | 2025-11-11 | 🎉 **jQWidgets 全面支持**：新增4种元素类型（复选框/单选框/开关/滑块）| +| **v2.4.1** | 2025-11-11 | 🔧 **高亮模块拆分**：element-highlighter 拆分5个子模块，更易维护 | +| **v2.4.0** | 2025-11-10 | ✨ **批量高亮 + 选择器测试**：
🎯 一键高亮所有元素或当前分类
🎛️ 9种选择器类型可选（智能/ID/XPath/CSS）
📊 实时统计反馈（成功/失败/不可用）
🧪 测试选择器可靠性和覆盖率 | +| **v2.3.0** | 2025-11-10 | 🎨 **覆盖层高亮**：
✨ 从 CSS class 改为固定定位覆盖层
🎯 解决元素被遮挡问题
🚀 适配 Google 等复杂页面
⚡ 60fps 脉冲动画 | +| **v2.2.0** | 2025-11-10 | 🔧 **模块化重构**：
✨ content.ts拆分7个模块（826→84行）
📦 单一职责原则
🎯 清晰分层架构
✅ 易于测试维护 | +| **v2.1.0** | 2025-11-05 | 🎯 **元素定位**：智能选择器生成、7级回退、元素高亮、一键复制 | +| **v2.0.0** | 2025-11-05 | 🎉 **重大重构**：React + TypeScript + MUI，现代化技术栈 | +| v1.2.0 | 2025-11-05 | 🎨 侧边栏UI | +| v1.0.0 | 2025-11-04 | 🚀 基础保存功能 | ## 🤝 贡献指南 diff --git a/selector.md b/selector.md new file mode 100644 index 0000000..e07cc09 --- /dev/null +++ b/selector.md @@ -0,0 +1,140 @@ +Web 开发 DOM 与选择器学习文档 +1. offset 在 DOM 中的含义 + +offset 是元素相对于某个参考点（通常是 offsetParent）的位置或尺寸。 + +常用属性： + +属性 含义 +offsetTop 元素顶部距离 offsetParent 顶部的距离 +offsetLeft 元素左边距离 offsetParent 左边的距离 +offsetWidth 元素布局宽度（内容 + 内边距 + 边框，不含外边距） +offsetHeight 元素布局高度 +offsetParent 最近的已定位父元素（position 不为 static） + +注意事项： + +offsetTop/Left 是相对 offsetParent 的。 + +想获取相对整个文档的坐标： + +const rect = el.getBoundingClientRect(); +const top = rect.top + window.pageYOffset; +const left = rect.left + window.pageXOffset; + + +offsetWidth/Height vs clientWidth/Height：client 不包含边框。 + +2. DOM 树索引是否唯一 + +DOM 本身没有全局唯一索引。 + +常见索引概念： + +父节点的子节点数组索引（parent.childNodes[index]） + +只在父节点下唯一。 + +NodeList 索引（document.querySelectorAll） + +只在 NodeList 内唯一。 + +自定义索引（data-index、id 等） + +可以作为全局唯一标识。 + +结论： + +DOM 树索引不是全局唯一。 + +想全局唯一标识节点，需要自己添加 id 或 data-* 属性。 + +3. 选择器（Selector） +3.1 CSS 选择器 + +用于选中元素并应用样式。 + +常见类型： + +选择器 示例 含义 +标签 p 选中所有

+ID #header 选中 id="header" 的元素 +类 .item 选中 class="item" 的元素 +组合 div.main p.text div.main 内的 p.text +属性 a[href^="https"] href 以 https 开头的 a +通配符 * 选中所有元素 +伪类 li:first-child 第一个 li +伪元素 p::after p 标签后生成内容 +3.2 JS 中的选择器 + +document.querySelector(selector) → 第一个匹配元素 + +document.querySelectorAll(selector) → 所有匹配元素 + +其他 DOM API： + +getElementById() + +getElementsByClassName() + +getElementsByTagName() + +4. XPath + +用于在 XML 或 HTML 文档中定位节点。 + +DOM 被看作一棵树，每个节点都可以通过路径表达式定位。 + +示例 HTML： + +

+ + +示例 XPath： + +XPath 含义 +/html/body/div/ul/li[2] 第二个 li（香蕉） +//li[@class='item'] 所有 class="item" 的 li +//li[text()='苹果'] 文本为 “苹果” 的 li +//div[@id='container'] id="container" 的 div + +XPath vs CSS Selector： + +XPath 可以上下左右遍历父子兄弟，CSS 只能向下。 + +XPath 支持文本匹配。 + +CSS 更易用，主要用于样式和 DOM 操作。 + +5. 控制台精准填写 元素 + +示例 ： + + + +方法 1：通过 id 定位（最精准） +const input = document.getElementById("jqxWidgetd183086b"); +input.value = "12345"; +input.dispatchEvent(new Event('input', { bubbles: true })); +input.dispatchEvent(new Event('change', { bubbles: true })); + +方法 2：通过类名或 data-name +const zrsInput = document.querySelector('input[data-name="ZRS"]'); +zrsInput.value = "12345"; +zrsInput.dispatchEvent(new Event('input', { bubbles: true })); + +方法 3：使用 XPath +const xpath = '//input[@data-name="ZRS"]'; +const input = document.evaluate(xpath, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue; +input.value = "12345"; +input.dispatchEvent(new Event('input', { bubbles: true })); + +方法 4：框架 API（如 jQWidgets / jQuery） +$('#jqxWidgetd183086b').val('12345'); +$('#jqxWidgetd183086b').trigger('change'); + + +⚠️ 注意：有些控件内部有双向绑定，直接改 DOM value 可能不会同步，需要触发事件或用框架 API。 \ No newline at end of file diff --git a/src/background/background.ts b/src/background/background.ts index 36d8648..394e20b 100644 --- a/src/background/background.ts +++ b/src/background/background.ts @@ -77,6 +77,42 @@ chrome.runtime.onMessage.addListener(( return true; // Async response } + // Forward highlightAllElements message to current tab + if (request.action === 'highlightAllElements') { + chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { + if (tabs[0]?.id) { + chrome.tabs.sendMessage(tabs[0].id, request, (response) => { + if (chrome.runtime.lastError) { + console.warn('Failed to send highlight all message:', chrome.runtime.lastError); + sendResponse({ success: false, error: chrome.runtime.lastError.message }); + } else { + sendResponse(response || { success: true }); + } + }); + } else { + sendResponse({ success: false, error: 'No active tab found' }); + } + }); + return true; // Async response + } + + // Forward clearAllHighlights message to current tab + if (request.action === 'clearAllHighlights') { + chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { + if (tabs[0]?.id) { + chrome.tabs.sendMessage(tabs[0].id, request, (response) => { + if (chrome.runtime.lastError) { + console.warn('Failed to send clear all highlights message:', chrome.runtime.lastError); + } + sendResponse(response || { success: true }); + }); + } else { + sendResponse({ success: true }); // Silently succeed if no tab + } + }); + return true; // Async response + } + // CORS proxy: use background fetch to bypass CORS restrictions if (request.action === 'fetchResource') { console.log('CORS proxy request:', request.url); @@ -128,6 +164,73 @@ chrome.runtime.onMessage.addListener(( return true; // Async response } + // Handle Viewer mode logs + if (request.action === 'viewerLog') { + const { logType, data } = request; + + switch (logType) { + case 'start': + console.log( + '%c🎬 [Viewer] 录制已开始', + 'background: #4CAF50; color: white; font-size: 14px; padding: 4px 12px; border-radius: 4px; font-weight: bold;' + ); + console.log(` 📋 会话 ID: ${data.sessionId}`); + console.log(` 🌐 URL: ${data.url}`); + console.log(` 📄 标题: ${data.title}`); + console.log(` ⏱️ 开始时间: ${new Date(data.startTime).toLocaleString()}`); + console.log('%c━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━', 'color: #4CAF50;'); + break; + + case 'action': + console.log( + `%c🎬 [Viewer] %c${data.count}. ${data.type}%c → ${data.target}`, + 'color: #4CAF50; font-weight: bold;', + 'color: #FF5722; font-weight: bold;', + 'color: #2196F3;' + ); + if (data.additionalInfo) { + console.log(` ↳ ${data.additionalInfo}`); + } + break; + + case 'summary': + console.log( + `%c📊 [Viewer] 已录制 ${data.count} 个操作 | 时长: ${data.duration}s`, + 'background: #E3F2FD; color: #1976D2; padding: 2px 8px; border-radius: 3px;' + ); + break; + + case 'pause': + console.log( + '%c⏸️ [Viewer] 录制已暂停', + 'background: #FF9800; color: white; padding: 4px 12px; border-radius: 4px; font-weight: bold;' + ); + console.log(` 📊 已录制: ${data.count} 个操作`); + break; + + case 'resume': + console.log( + '%c▶️ [Viewer] 录制已继续', + 'background: #4CAF50; color: white; padding: 4px 12px; border-radius: 4px; font-weight: bold;' + ); + console.log(` 📊 当前进度: ${data.count} 个操作`); + break; + + case 'stop': + console.log( + '%c⏹️ [Viewer] 录制已停止', + 'background: #2196F3; color: white; font-size: 14px; padding: 4px 12px; border-radius: 4px; font-weight: bold;' + ); + console.log(` 📊 总操作数: ${data.totalActions}`); + console.log(` ⏱️ 总时长: ${data.duration}s`); + console.log(` 📋 会话 ID: ${data.sessionId}`); + console.log('%c━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━', 'color: #2196F3;'); + break; + } + + return false; // No response needed + } + return true; // Keep message channel open }); diff --git a/src/content/content.ts b/src/content/content.ts index 3dba718..6d13858 100644 --- a/src/content/content.ts +++ b/src/content/content.ts @@ -1,15 +1,25 @@ -// Content Script - TypeScript port from content.js - -import type { - ChromeMessage, - MessageResponse, - SaveOptions, - SaveResult, - ElementData, - PageStats, - ProgressUpdate, - ElementSelectors -} from '../sidepanel/types'; +// Content Script Entry Point - Refactored for modularity +import type { ChromeMessage, MessageResponse, ProgressUpdate } from '../sidepanel/types'; +import { extractPageContent } from './page-extractor'; +import { extractInteractiveElements } from './element-extractor'; +import { highlightElement, clearHighlight, highlightAllElements, clearAllHighlights } from './element-highlighter/index'; +import { + startRecording, + pauseRecording, + resumeRecording, + stopRecording, + getRecordingState, + getCurrentSession, + exportRecording, + clearRecording +} from './viewer-recorder/index'; +import { + loadPlaybackSession, + startPlayback, + pausePlayback, + stopPlayback, + seekPlayback +} from './viewer-player/index'; // Listen for messages chrome.runtime.onMessage.addListener(( @@ -84,742 +94,218 @@ chrome.runtime.onMessage.addListener(( return true; } - return false; // Don't keep channel open for unknown actions -}); - -async function extractPageContent( - options: SaveOptions, - onProgress: (progress: ProgressUpdate) => void -): Promise { - const stats: PageStats = { images: 0, css: 0, icons: 0, scripts: 0, failed: 0, canvas: 0, trackersRemoved: 0 }; - - // Remove trackers - if (options.removeTrackers) { - const removed = removeTrackers(); - stats.trackersRemoved = removed; - onProgress({ - text: `已移除 ${removed} 个追踪器`, - percent: 10 - }); - } - - // Process images - if (options.includeImages) { - const imgs = Array.from(document.querySelectorAll("img")); - for (let i = 0; i < imgs.length; i++) { - const img = imgs[i] as HTMLImageElement; - try { - const src = getRealSrc(img, 'src'); - if (src && !src.startsWith('data:') && !src.startsWith('blob:')) { - const dataURL = await toDataURL(src, 5000); - if (dataURL) { - img.src = dataURL; - img.removeAttribute('data-src'); - img.removeAttribute('data-lazy-src'); - img.removeAttribute('data-original'); - stats.images++; - } else { - stats.failed++; - } - } - // Update progress every 5 images - if ((i + 1) % 5 === 0) { - onProgress({ - text: `正在获取图片 ${i + 1}/${imgs.length}`, - percent: Math.min(20 + (stats.images / imgs.length * 30), 50) - }); - } - } catch (e) { - stats.failed++; - } + // Handle highlight all elements request + if (request.action === 'highlightAllElements') { + console.log('Received highlight all elements request'); + + try { + const result = highlightAllElements( + request.elements, + request.selectorType || 'auto' + ); + sendResponse({ success: true, result }); + } catch (error) { + console.error('Highlight all failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } + + return true; } - // Process CSS - if (options.includeCSS) { - const links = Array.from(document.querySelectorAll('link[rel="stylesheet"]')); - for (let i = 0; i < links.length; i++) { - const link = links[i] as HTMLLinkElement; - try { - onProgress({ - text: `正在处理样式表 ${i + 1}/${links.length}`, - percent: 50 + (i / links.length * 15) - }); - - const res = await Promise.race([ - fetch(link.href), - new Promise((_, reject) => setTimeout(() => reject(new Error('Timeout')), 5000)) - ]); - const css = await res.text(); - - // Process URLs in CSS - let processedCSS = css; - const urlRegex = /url\(['"]?(.*?)['"]?\)/g; - const matches = Array.from(css.matchAll(urlRegex)); - - for (const match of matches) { - const url = match[1]; - if (!url.startsWith('data:') && !url.startsWith('blob:') && url.trim()) { - try { - const absoluteUrl = new URL(url, link.href).href; - const dataURL = await toDataURL(absoluteUrl, 3000); - if (dataURL) { - processedCSS = processedCSS.replace(match[0], `url("${dataURL}")`); - } - } catch {} - } - } - - const style = document.createElement("style"); - style.textContent = processedCSS; - link.replaceWith(style); - stats.css++; - } catch { - stats.failed++; - } + // Handle clear all highlights request + if (request.action === 'clearAllHighlights') { + console.log('Received clear all highlights request'); + + try { + clearAllHighlights(); + sendResponse({ success: true }); + } catch (error) { + console.error('Clear all highlights failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } + + return true; } - onProgress({ - text: '正在生成HTML...', - percent: 85 - }); - - // Generate HTML - const doctype = ''; - let html = document.documentElement.outerHTML; - - const metaInfo = ` - - - - - `; - - html = html.replace('', '' + metaInfo); - const fullHTML = doctype + '\n' + html; - - const blob = new Blob([fullHTML], { type: 'text/html;charset=utf-8' }); - const dataUrl = URL.createObjectURL(blob); - - onProgress({ - text: '准备下载...', - percent: 95 - }); - - return { - success: true, - dataUrl: dataUrl, - filename: `${document.title || 'page'}-${Date.now()}.html`, - size: `${(blob.size / 1024 / 1024).toFixed(2)} MB`, - stats: stats - }; -} - -// Helper functions -async function toDataURL(url: string, timeout = 5000): Promise { - try { - return await Promise.race([ - (async () => { - const res = await fetch(url); - const blob = await res.blob(); - return await new Promise(resolve => { - const reader = new FileReader(); - reader.onload = () => resolve(reader.result as string); - reader.onerror = () => resolve(null); - reader.readAsDataURL(blob); - }); - })(), - new Promise(resolve => setTimeout(() => resolve(null), timeout)) - ]); - } catch { - return null; - } -} - -function getRealSrc(element: HTMLElement, attrName = 'src'): string { - const possibleAttrs = [ - attrName, - `data-${attrName}`, - 'data-lazy-src', - 'data-original', - 'data-lazy', - 'data-lazy-load', - ]; - - for (const attr of possibleAttrs) { - const value = element.getAttribute(attr); - if (value && value.trim() && !value.startsWith('data:') && !value.startsWith('blob:')) { - return value; - } - } - return (element as any)[attrName]; -} + // ==================== Viewer Recording Messages ==================== -function removeTrackers(): number { - const TRACKER_PATTERNS = [ - /google-analytics\.com/i, - /googletagmanager\.com/i, - /baidu\.com\/hm\.js/i, - /cnzz\.com/i, - /facebook\.net\/fbevents/i, - /doubleclick\.net/i, - /scorecardresearch\.com/i, - ]; - - let removed = 0; - const scripts = document.querySelectorAll('script[src]'); - scripts.forEach(script => { - const src = script.getAttribute('src'); - if (src && TRACKER_PATTERNS.some(pattern => pattern.test(src))) { - script.remove(); - removed++; - } - }); - - const inlineScripts = document.querySelectorAll('script:not([src])'); - inlineScripts.forEach(script => { - const content = script.textContent || ''; - if (TRACKER_PATTERNS.some(pattern => pattern.test(content))) { - script.remove(); - removed++; - } - }); - - return removed; -} - -// ==================== Extract Interactive Elements Function ==================== - -function extractInteractiveElements(): ElementData { - const elements = { - buttons: [] as any[], - links: [] as any[], - forms: [] as any[], - inputs: [] as any[], - selects: [] as any[], - textareas: [] as any[], - custom: [] as any[] - }; - - function extractDataAttributes(el: Element): string { - const data: Record = {}; - if ('dataset' in el && el.dataset) { - const dataset = el.dataset as DOMStringMap; - Object.keys(dataset).forEach(key => { - data[key] = dataset[key] || ''; + // Handle start recording + if (request.action === 'startRecording') { + console.log('Received start recording request'); + + startRecording() + .then(() => { + sendResponse({ success: true }); + }) + .catch((error) => { + console.error('Start recording failed:', error); + sendResponse({ success: false, error: (error as Error).message }); }); - } - return Object.keys(data).length > 0 ? JSON.stringify(data) : ''; + + return true; } - function findAssociatedLabel(input: HTMLElement): string { - if (input.id) { - const label = document.querySelector(`label[for="${input.id}"]`); - if (label) return label.textContent?.trim() || ''; - } + // Handle pause recording + if (request.action === 'pauseRecording') { + console.log('Received pause recording request'); - const parentLabel = input.closest('label'); - if (parentLabel) { - return parentLabel.textContent?.trim() || ''; + try { + pauseRecording(); + sendResponse({ success: true }); + } catch (error) { + console.error('Pause recording failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } - return ''; - } - - // Selector generation functions - function getXPath(element: Element): string { - if (element.id) return `//*[@id="${element.id}"]`; - const parts: string[] = []; - let currentElement: Element | null = element; - while (currentElement && currentElement.nodeType === Node.ELEMENT_NODE) { - let index = 0; - let sibling: Element | null = currentElement.previousElementSibling; - while (sibling) { - if (sibling.nodeName === currentElement.nodeName) index++; - sibling = sibling.previousElementSibling; - } - const tagName = currentElement.nodeName.toLowerCase(); - const pathIndex = index > 0 ? `[${index + 1}]` : ''; - parts.unshift(tagName + pathIndex); - currentElement = currentElement.parentElement; - } - return parts.length ? '/' + parts.join('/') : ''; + return true; } - function getShortXPath(element: Element): string | undefined { - if (element.id) return `//*[@id="${element.id}"]`; - let current: Element | null = element; - const parts: string[] = []; - while (current && current !== document.documentElement) { - if (current.id) { - parts.unshift(`//*[@id="${current.id}"]`); - return parts.join('/') || undefined; - } - let index = 0; - let sibling: Element | null = current.previousElementSibling; - while (sibling) { - if (sibling.nodeName === current.nodeName) index++; - sibling = sibling.previousElementSibling; - } - const tagName = current.nodeName.toLowerCase(); - const pathIndex = index > 0 ? `[${index + 1}]` : ''; - parts.unshift(tagName + pathIndex); - current = current.parentElement; + // Handle resume recording + if (request.action === 'resumeRecording') { + console.log('Received resume recording request'); + + try { + resumeRecording(); + sendResponse({ success: true }); + } catch (error) { + console.error('Resume recording failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } - return undefined; + + return true; } - function generateSelectors(element: HTMLElement, type: string, index: number): { selectors: ElementSelectors; extractId: string } { - const extractId = `extract-${type}-${index}`; - element.setAttribute('data-extract-id', extractId); + // Handle stop recording + if (request.action === 'stopRecording') { + console.log('Received stop recording request'); - const selectors: ElementSelectors = { - xpath: getXPath(element), - xpathShort: getShortXPath(element), - cssById: element.id ? `#${CSS.escape(element.id)}` : undefined, - cssByClass: element.className && typeof element.className === 'string' ? - `${element.tagName.toLowerCase()}${element.className.trim().split(/\s+/).filter(c => c.length > 0).map(c => `.${CSS.escape(c)}`).join('')}` : undefined, - cssByAttribute: (() => { - const attrs: string[] = []; - if (element.getAttribute('name')) attrs.push(`[name="${CSS.escape(element.getAttribute('name')!)}"]`); - if (element.getAttribute('type')) attrs.push(`[type="${CSS.escape(element.getAttribute('type')!)}"]`); - return attrs.length > 0 ? `${element.tagName.toLowerCase()}${attrs.join('')}` : undefined; - })(), - cssByNthChild: (() => { - const parent = element.parentElement; - if (!parent) return element.tagName.toLowerCase(); - const siblings = Array.from(parent.children); - const index = siblings.indexOf(element) + 1; - return `${element.tagName.toLowerCase()}:nth-child(${index})`; - })(), - cssCombined: (() => { - const parts: string[] = [element.tagName.toLowerCase()]; - if (element.id) { - parts.push(`#${CSS.escape(element.id)}`); - return parts.join(''); - } - if (element.className && typeof element.className === 'string') { - const classes = element.className.trim().split(/\s+/).filter(c => c.length > 0); - if (classes.length > 0) parts.push(`.${CSS.escape(classes[0])}`); - } - if (element.getAttribute('name')) parts.push(`[name="${CSS.escape(element.getAttribute('name')!)}"]`); - return parts.length > 1 ? parts.join('') : undefined; - })() - }; + stopRecording() + .then((session) => { + sendResponse({ success: true, data: session }); + }) + .catch((error) => { + console.error('Stop recording failed:', error); + sendResponse({ success: false, error: (error as Error).message }); + }); - return { selectors, extractId }; + return true; } - try { - // Extract buttons - let buttonIndex = 0; - document.querySelectorAll('button, [role="button"], input[type="button"], input[type="submit"], input[type="reset"]').forEach(el => { - const element = el as HTMLElement; - if (element.offsetHeight > 0 && element.offsetWidth > 0) { - const inputEl = el as HTMLInputElement; - const { selectors, extractId } = generateSelectors(element, 'button', buttonIndex++); - elements.buttons.push({ - type: 'button', - text: element.textContent?.trim() || inputEl.value || element.getAttribute('aria-label') || '', - id: element.id || '', - class: element.className || '', - tag: element.tagName.toLowerCase(), - type_specific: (el as any).type || '', - onclick: 'onclick' in element && (element as any).onclick ? 'yes' : 'no', - data_attributes: extractDataAttributes(element), - aria_label: element.getAttribute('aria-label') || '', - title: element.title || '', - _selectors: selectors, - _extractId: extractId - }); - } - }); - - // Extract links - let linkIndex = 0; - document.querySelectorAll('a[href]').forEach(el => { - const element = el as HTMLAnchorElement; - if (element.offsetHeight > 0 && element.offsetWidth > 0) { - const { selectors, extractId } = generateSelectors(element, 'link', linkIndex++); - elements.links.push({ - type: 'link', - text: element.textContent?.trim() || '', - href: element.href || element.getAttribute('href') || '', - id: element.id || '', - class: element.className || '', - target: element.target || '', - title: element.title || '', - data_attributes: extractDataAttributes(element), - aria_label: element.getAttribute('aria-label') || '', - _selectors: selectors, - _extractId: extractId - }); - } - }); - - // Extract forms - let formIndex = 0; - document.querySelectorAll('form').forEach(form => { - const formEl = form as HTMLFormElement; - if (formEl.offsetHeight > 0 && formEl.offsetWidth > 0) { - const { selectors, extractId } = generateSelectors(formEl, 'form', formIndex++); - elements.forms.push({ - type: 'form', - id: formEl.id || '', - name: formEl.name || '', - method: formEl.method || 'GET', - action: formEl.action || '', - class: formEl.className || '', - fields_count: formEl.querySelectorAll('input, textarea, select').length, - submit_button: formEl.querySelector('button[type="submit"], input[type="submit"]')?.textContent?.trim() || '提交', - data_attributes: extractDataAttributes(formEl), - _selectors: selectors, - _extractId: extractId - }); - } - }); - - // Extract inputs - let inputIndex = 0; - document.querySelectorAll('input:not([type="hidden"])').forEach(el => { - const input = el as HTMLInputElement; - if (input.offsetHeight > 0 && input.offsetWidth > 0) { - const rect = input.getBoundingClientRect(); - if (rect.width > 0 && rect.height > 0) { - const { selectors, extractId } = generateSelectors(input, 'input', inputIndex++); - elements.inputs.push({ - type: 'input', - input_type: input.type || 'text', - name: input.name || '', - id: input.id || '', - placeholder: input.placeholder || '', - value: input.value ? input.value.substring(0, 50) : '', - required: input.required ? 'yes' : 'no', - disabled: input.disabled ? 'yes' : 'no', - class: input.className || '', - label: findAssociatedLabel(input), - data_attributes: extractDataAttributes(input), - min: input.min || '', - max: input.max || '', - pattern: input.pattern || '', - _selectors: selectors, - _extractId: extractId - }); - } - } - }); - - // Extract selects - let selectIndex = 0; - document.querySelectorAll('select').forEach(el => { - const select = el as HTMLSelectElement; - if (select.offsetHeight > 0 && select.offsetWidth > 0) { - const options: string[] = []; - select.querySelectorAll('option').forEach(opt => { - options.push(opt.textContent?.trim() || ''); - }); - - const { selectors, extractId } = generateSelectors(select, 'select', selectIndex++); - elements.selects.push({ - type: 'select', - name: select.name || '', - id: select.id || '', - options_count: options.length, - options: options.slice(0, 10).join(' | '), - required: select.required ? 'yes' : 'no', - disabled: select.disabled ? 'yes' : 'no', - class: select.className || '', - label: findAssociatedLabel(select), - data_attributes: extractDataAttributes(select), - _selectors: selectors, - _extractId: extractId - }); - } - }); - - // Extract textareas - let textareaIndex = 0; - document.querySelectorAll('textarea').forEach(el => { - const textarea = el as HTMLTextAreaElement; - if (textarea.offsetHeight > 0 && textarea.offsetWidth > 0) { - const { selectors, extractId } = generateSelectors(textarea, 'textarea', textareaIndex++); - elements.textareas.push({ - type: 'textarea', - name: textarea.name || '', - id: textarea.id || '', - placeholder: textarea.placeholder || '', - rows: textarea.rows || '', - cols: textarea.cols || '', - required: textarea.required ? 'yes' : 'no', - disabled: textarea.disabled ? 'yes' : 'no', - class: textarea.className || '', - label: findAssociatedLabel(textarea), - data_attributes: extractDataAttributes(textarea), - _selectors: selectors, - _extractId: extractId - }); - } - }); - - // Extract custom components - let customIndex = 0; - document.querySelectorAll('[data-toggle], [onclick]:not(script), [role="tab"], [role="menuitem"], [data-action], [data-modal]').forEach(el => { - const element = el as HTMLElement; - if (element.offsetHeight > 0 && element.offsetWidth > 0 && - !element.querySelector('input, button, a, textarea, select')) { - - const customType = element.getAttribute('data-toggle') || - element.getAttribute('role') || - element.getAttribute('data-action') || - 'custom'; - - const { selectors, extractId } = generateSelectors(element, 'custom', customIndex++); - elements.custom.push({ - type: customType, - text: element.textContent?.trim()?.substring(0, 100) || '', - tag: element.tagName.toLowerCase(), - id: element.id || '', - class: element.className || '', - onclick: 'onclick' in element && (element as any).onclick ? 'yes' : 'no', - data_attributes: extractDataAttributes(element), - aria_label: element.getAttribute('aria-label') || '', - title: element.title || '', - _selectors: selectors, - _extractId: extractId + // Handle get recording state + if (request.action === 'getRecordingState') { + getCurrentSession() + .then((session) => { + sendResponse({ + success: true, + data: { + state: getRecordingState(), + session + } }); - } - }); + }) + .catch((error) => { + console.error('Get recording state failed:', error); + sendResponse({ success: false, error: (error as Error).message }); + }); + + return true; + } - return { - success: true, - url: window.location.href, - title: document.title, - timestamp: new Date().toISOString(), - elements: elements as any, - summary: { - total: Object.values(elements).reduce((sum, arr) => sum + arr.length, 0), - buttons: elements.buttons.length, - links: elements.links.length, - forms: elements.forms.length, - inputs: elements.inputs.length, - selects: elements.selects.length, - textareas: elements.textareas.length, - custom: elements.custom.length - } - }; - } catch (error) { - return { - success: false, - url: window.location.href, - title: document.title, - timestamp: new Date().toISOString(), - error: (error as Error).message, - elements: elements as any, - summary: { - total: 0, - buttons: 0, - links: 0, - forms: 0, - inputs: 0, - selects: 0, - textareas: 0, - custom: 0 - } - }; + // Handle export recording + if (request.action === 'exportRecording') { + exportRecording() + .then(() => { + sendResponse({ success: true }); + }) + .catch((error) => { + console.error('Export recording failed:', error); + sendResponse({ success: false, error: (error as Error).message }); + }); + + return true; } -} -// ==================== Element Highlighting Functionality ==================== + // Handle clear recording + if (request.action === 'clearRecording') { + clearRecording() + .then(() => { + sendResponse({ success: true }); + }) + .catch((error) => { + console.error('Clear recording failed:', error); + sendResponse({ success: false, error: (error as Error).message }); + }); + + return true; + } -let currentHighlightedElement: HTMLElement | null = null; -let highlightTimeout: ReturnType | null = null; -const HIGHLIGHT_CLASS = 'singlefile-lite-highlight'; -const HIGHLIGHT_STYLE_ID = 'singlefile-lite-highlight-style'; + // ==================== Viewer Playback Messages ==================== -// Inject highlight styles into page -function injectHighlightStyles(): void { - if (document.getElementById(HIGHLIGHT_STYLE_ID)) return; - - const style = document.createElement('style'); - style.id = HIGHLIGHT_STYLE_ID; - style.textContent = ` - .${HIGHLIGHT_CLASS} { - outline: 3px solid #ff4081 !important; - box-shadow: 0 0 20px rgba(255, 64, 129, 0.8) !important; - background-color: rgba(255, 255, 0, 0.15) !important; - animation: singlefile-lite-pulse 2s ease-in-out infinite !important; - transition: all 0.3s ease !important; - position: relative !important; - z-index: 999999 !important; - } + // Handle start playback + if (request.action === 'startPlayback') { + console.log('Received start playback request'); - @keyframes singlefile-lite-pulse { - 0%, 100% { - box-shadow: 0 0 20px rgba(255, 64, 129, 0.8); - } - 50% { - box-shadow: 0 0 30px rgba(255, 64, 129, 1); - } - } - `; - document.head.appendChild(style); -} - -// Locate element using selectors with fallback logic -function locateElement(selectors: ElementSelectors, extractId?: string): HTMLElement | null { - // Try 1: data-extract-id (most reliable) - if (extractId) { - const element = document.querySelector(`[data-extract-id="${extractId}"]`); - if (element) { - console.log('Element found by extract-id:', extractId); - return element as HTMLElement; - } - } - - // Try 2: CSS by ID (highly reliable) - if (selectors.cssById) { - try { - const element = document.querySelector(selectors.cssById); - if (element) { - console.log('Element found by CSS ID:', selectors.cssById); - return element as HTMLElement; - } - } catch (e) { - console.warn('CSS by ID failed:', e); - } - } - - // Try 3: XPath (full or short) - const xpathToTry = selectors.xpathShort || selectors.xpath; - if (xpathToTry) { - try { - const result = document.evaluate( - xpathToTry, - document, - null, - XPathResult.FIRST_ORDERED_NODE_TYPE, - null - ); - if (result.singleNodeValue) { - console.log('Element found by XPath:', xpathToTry); - return result.singleNodeValue as HTMLElement; - } - } catch (e) { - console.warn('XPath failed:', e); - } - } - - // Try 4: CSS Combined - if (selectors.cssCombined) { try { - const element = document.querySelector(selectors.cssCombined); - if (element) { - console.log('Element found by CSS combined:', selectors.cssCombined); - return element as HTMLElement; - } - } catch (e) { - console.warn('CSS combined failed:', e); + const { session, speed } = request as any; + loadPlaybackSession(session, speed || 1); + startPlayback(); + sendResponse({ success: true }); + } catch (error) { + console.error('Start playback failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } + + return true; } - - // Try 5: CSS by Class - if (selectors.cssByClass) { + + // Handle pause playback + if (request.action === 'pausePlayback') { + console.log('Received pause playback request'); + try { - const element = document.querySelector(selectors.cssByClass); - if (element) { - console.log('Element found by CSS class:', selectors.cssByClass); - return element as HTMLElement; - } - } catch (e) { - console.warn('CSS by class failed:', e); + pausePlayback(); + sendResponse({ success: true }); + } catch (error) { + console.error('Pause playback failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } + + return true; } - - // Try 6: CSS by Attribute - if (selectors.cssByAttribute) { + + // Handle stop playback + if (request.action === 'stopPlayback') { + console.log('Received stop playback request'); + try { - const element = document.querySelector(selectors.cssByAttribute); - if (element) { - console.log('Element found by CSS attribute:', selectors.cssByAttribute); - return element as HTMLElement; - } - } catch (e) { - console.warn('CSS by attribute failed:', e); + stopPlayback(); + sendResponse({ success: true }); + } catch (error) { + console.error('Stop playback failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } + + return true; } - - // Try 7: CSS nth-child (least reliable, but better than nothing) - if (selectors.cssByNthChild) { + + // Handle seek playback + if (request.action === 'seekPlayback') { + console.log('Received seek playback request'); + try { - const element = document.querySelector(selectors.cssByNthChild); - if (element) { - console.log('Element found by CSS nth-child:', selectors.cssByNthChild); - return element as HTMLElement; - } - } catch (e) { - console.warn('CSS nth-child failed:', e); + const { time } = request as any; + seekPlayback(time); + sendResponse({ success: true }); + } catch (error) { + console.error('Seek playback failed:', error); + sendResponse({ success: false, error: (error as Error).message }); } + + return true; } - - console.warn('Element not found with any selector'); - return null; -} - -// Highlight an element -function highlightElement(selectors: ElementSelectors, extractId?: string): boolean { - // Clear previous highlight - clearHighlight(); - - // Inject styles if not already present - injectHighlightStyles(); - - // Locate element - const element = locateElement(selectors, extractId); - if (!element) { - console.warn('Could not locate element to highlight'); - return false; - } - - // Scroll element into view - try { - element.scrollIntoView({ - behavior: 'smooth', - block: 'center', - inline: 'center' - }); - } catch (e) { - // Fallback for older browsers - element.scrollIntoView(); - } - - // Add highlight class - element.classList.add(HIGHLIGHT_CLASS); - currentHighlightedElement = element; - - // Auto-clear after 5 seconds - highlightTimeout = setTimeout(() => { - clearHighlight(); - }, 5000); - - console.log('Element highlighted successfully'); - return true; -} -// Clear highlight -function clearHighlight(): void { - if (currentHighlightedElement) { - currentHighlightedElement.classList.remove(HIGHLIGHT_CLASS); - currentHighlightedElement = null; - } - - if (highlightTimeout) { - clearTimeout(highlightTimeout); - highlightTimeout = null; - } - - console.log('Highlight cleared'); -} + return false; // Don't keep channel open for unknown actions +}); console.log('SingleFile Lite content script loaded'); diff --git a/src/content/element-extractor.ts b/src/content/element-extractor.ts new file mode 100644 index 0000000..67e0983 --- /dev/null +++ b/src/content/element-extractor.ts @@ -0,0 +1,692 @@ +// Interactive element extraction functionality +import type { ElementData } from '../sidepanel/types'; +import { generateSelectors } from './selectors/selector-generator'; +import { extractDataAttributes, findAssociatedLabel } from './utils/resource-utils'; + +/** + * Extract all interactive elements from the page + * Returns categorized elements with selectors for location + */ +export function extractInteractiveElements(): ElementData { + // 🔥 清除所有旧的 Extract ID（解决 SPA 页面切换时的残留问题） + console.log('🧹 Cleaning old extract IDs before extraction...'); + const oldIds = document.querySelectorAll('[data-extract-id]'); + if (oldIds.length > 0) { + console.log(`Found ${oldIds.length} old extract IDs, removing...`); + oldIds.forEach(el => el.removeAttribute('data-extract-id')); + } + + const elements = { + buttons: [] as any[], + links: [] as any[], + forms: [] as any[], + inputs: [] as any[], + selects: [] as any[], + textareas: [] as any[], + checkboxes: [] as any[], + radios: [] as any[], + switches: [] as any[], + sliders: [] as any[], + datatables: [] as any[], + custom: [] as any[] + }; + + try { + // Extract buttons (including button-styled links and jQWidgets buttons) + let buttonIndex = 0; + document.querySelectorAll('button, [role="button"], input[type="button"], input[type="submit"], input[type="reset"], a.btn, a.button, [xtype="button"], .jqx-button, a[class*="bh-btn"], a.ant-btn, a.el-button, a.mui-btn, a[class*="-button"], a[class*="btn-"], a[class*="btn"]').forEach(el => { + const element = el as HTMLElement; + // More lenient visibility check - accept elements that might be temporarily hidden + // or have very small dimensions due to animations/transitions + const rect = element.getBoundingClientRect(); + const isVisibleEnough = element.offsetHeight > 0 || element.offsetWidth > 0 || rect.height > 0 || rect.width > 0; + + if (isVisibleEnough) { + const inputEl = el as HTMLInputElement; + const { selectors, extractId } = generateSelectors(element, 'button', buttonIndex++); + elements.buttons.push({ + type: 'button', + text: element.textContent?.trim() || inputEl.value || element.getAttribute('aria-label') || '', + id: element.id || '', + class: element.className || '', + tag: element.tagName.toLowerCase(), + type_specific: (el as any).type || '', + onclick: 'onclick' in element && (element as any).onclick ? 'yes' : 'no', + data_attributes: extractDataAttributes(element), + aria_label: element.getAttribute('aria-label') || '', + title: element.title || '', + _selectors: selectors, + _extractId: extractId + }); + } + }); + + // Extract links (excluding button-styled links) + let linkIndex = 0; + document.querySelectorAll('a[href], a[url]').forEach(el => { + const element = el as HTMLAnchorElement; + if (element.offsetHeight > 0 && element.offsetWidth > 0) { + // Skip button-styled links (already extracted as buttons) + const classList = element.className || ''; + if (classList.includes('btn') || + classList.includes('button') || + classList.includes('-btn') || + element.getAttribute('role') === 'button') { + return; + } + + const { selectors, extractId } = generateSelectors(element, 'link', linkIndex++); + elements.links.push({ + type: 'link', + text: element.textContent?.trim() || '', + href: element.href || element.getAttribute('href') || element.getAttribute('url') || '', + id: element.id || '', + class: element.className || '', + target: element.target || '', + title: element.title || '', + data_attributes: extractDataAttributes(element), + aria_label: element.getAttribute('aria-label') || '', + _selectors: selectors, + _extractId: extractId + }); + } + }); + + // Extract forms + let formIndex = 0; + document.querySelectorAll('form').forEach(form => { + const formEl = form as HTMLFormElement; + if (formEl.offsetHeight > 0 && formEl.offsetWidth > 0) { + const { selectors, extractId } = generateSelectors(formEl, 'form', formIndex++); + elements.forms.push({ + type: 'form', + id: formEl.id || '', + name: formEl.name || '', + method: formEl.method || 'GET', + action: formEl.action || '', + class: formEl.className || '', + fields_count: formEl.querySelectorAll('input, textarea, select').length, + submit_button: formEl.querySelector('button[type="submit"], input[type="submit"]')?.textContent?.trim() || '提交', + data_attributes: extractDataAttributes(formEl), + _selectors: selectors, + _extractId: extractId + }); + } + }); + + // Extract inputs + let inputIndex = 0; + document.querySelectorAll('input:not([type="hidden"])').forEach(el => { + const input = el as HTMLInputElement; + if (input.offsetHeight > 0 && input.offsetWidth > 0) { + const rect = input.getBoundingClientRect(); + if (rect.width > 0 && rect.height > 0) { + const { selectors, extractId } = generateSelectors(input, 'input', inputIndex++); + elements.inputs.push({ + type: 'input', + input_type: input.type || 'text', + name: input.name || '', + id: input.id || '', + placeholder: input.placeholder || '', + value: input.value ? input.value.substring(0, 50) : '', + required: input.required ? 'yes' : 'no', + disabled: input.disabled ? 'yes' : 'no', + class: input.className || '', + label: findAssociatedLabel(input), + data_attributes: extractDataAttributes(input), + min: input.min || '', + max: input.max || '', + pattern: input.pattern || '', + _selectors: selectors, + _extractId: extractId + }); + } + } + }); + + // Extract jQWidgets custom text inputs (xtype="text", etc.) + document.querySelectorAll('[xtype="text"], [xtype="textarea"], [xtype="number"], [xtype="password"], [xtype="email"], [xtype="tel"], [xtype="date"], [xtype="date-local"], [xtype="datetime"], [xtype="datetime-local"], [xtype="time"], [xtype="month"], [xtype="week"]').forEach(el => { + const element = el as HTMLElement; + + // Skip if it's a native input or textarea + if (element.tagName.toLowerCase() === 'input' || element.tagName.toLowerCase() === 'textarea') return; + + // Skip if invisible + if (element.offsetHeight === 0 || element.offsetWidth === 0) return; + + const xtype = element.getAttribute('xtype') || 'text'; + const currentValue = element.textContent?.trim() || element.getAttribute('value') || ''; + + const { selectors, extractId } = generateSelectors(element, 'input', inputIndex++); + elements.inputs.push({ + type: 'input-custom', + input_type: xtype, // text, textarea, number, password, email, tel + name: element.getAttribute('data-name') || element.getAttribute('name') || '', + id: element.id || '', + placeholder: element.getAttribute('placeholder') || element.getAttribute('data-placeholder') || '', + value: currentValue ? currentValue.substring(0, 50) : '', + required: element.getAttribute('data-required') === 'true' || element.getAttribute('aria-required') === 'true' ? 'yes' : 'no', + disabled: element.getAttribute('data-disabled') === 'true' || element.getAttribute('aria-disabled') === 'true' ? 'yes' : 'no', + class: element.className || '', + label: element.getAttribute('data-caption') || + element.getAttribute('aria-label') || + element.getAttribute('data-label') || + findAssociatedLabel(element), + data_attributes: extractDataAttributes(element), + min: element.getAttribute('min') || '', + max: element.getAttribute('max') || '', + pattern: element.getAttribute('pattern') || '', + _selectors: selectors, + _extractId: extractId + }); + }); + + // Extract checkboxes (native and jQWidgets) + let checkboxIndex = 0; + + // Native checkboxes + document.querySelectorAll('input[type="checkbox"]').forEach(el => { + const checkbox = el as HTMLInputElement; + if (checkbox.offsetHeight > 0 || checkbox.offsetWidth > 0) { + const { selectors, extractId } = generateSelectors(checkbox, 'checkbox', checkboxIndex++); + elements.checkboxes.push({ + type: 'checkbox', + name: checkbox.name || '', + id: checkbox.id || '', + value: checkbox.value || '', + checked: checkbox.checked ? 'yes' : 'no', + required: checkbox.required ? 'yes' : 'no', + disabled: checkbox.disabled ? 'yes' : 'no', + class: checkbox.className || '', + label: findAssociatedLabel(checkbox), + data_attributes: extractDataAttributes(checkbox), + _selectors: selectors, + _extractId: extractId + }); + } + }); + + // jQWidgets custom checkboxes (including ListBox checkboxes) + document.querySelectorAll('[xtype="checkbox"], .jqx-checkbox, .jqx-checkbox-default, [role="checkbox"]:not(input)').forEach(el => { + const element = el as HTMLElement; + + // Skip if it's a native checkbox + if (element.tagName.toLowerCase() === 'input') return; + + // More strict visibility check (check computed style) + const rect = element.getBoundingClientRect(); + const style = window.getComputedStyle(element); + const isVisible = ( + (element.offsetHeight > 0 || element.offsetWidth > 0 || rect.width > 0 || rect.height > 0) && + style.visibility !== 'hidden' && + style.display !== 'none' + ); + + if (!isVisible) return; + + // Check if checked (multiple ways for different jQWidgets patterns) + const isChecked = element.getAttribute('aria-checked') === 'true' || + element.getAttribute('data-checked') === 'true' || + element.classList.contains('jqx-checkbox-check-checked') || + element.querySelector('.jqx-checkbox-check-checked') !== null || + element.querySelector('span.checkBoxCheck') !== null; + + // Try to find associated label + let label = element.getAttribute('data-caption') || + element.getAttribute('aria-label') || + element.getAttribute('data-label') || + findAssociatedLabel(element); + + // Special handling for jQWidgets ListBox checkboxes + if (!label) { + // Find parent .chkbox container, then find sibling span with text + const chkboxContainer = element.closest('.chkbox'); + if (chkboxContainer && chkboxContainer.parentElement) { + const listItemElement = chkboxContainer.parentElement; + const labelSpan = listItemElement.querySelector('span.jqx-listitem-state-normal, span.jqx-item'); + if (labelSpan) { + label = labelSpan.textContent?.trim() || ''; + } + } + } + + // If still no label, try from nearest [role="option"] parent + if (!label) { + const optionElement = element.closest('[role="option"]'); + if (optionElement) { + const labelSpan = optionElement.querySelector('span.jqx-listitem-state-normal, span.jqx-item'); + if (labelSpan) { + label = labelSpan.textContent?.trim() || ''; + } + } + } + + const { selectors, extractId } = generateSelectors(element, 'checkbox', checkboxIndex++); + elements.checkboxes.push({ + type: 'checkbox-custom', + name: element.getAttribute('data-name') || element.getAttribute('name') || '', + id: element.id || '', + value: element.getAttribute('value') || element.getAttribute('data-value') || '', + checked: isChecked ? 'yes' : 'no', + required: element.getAttribute('data-required') === 'true' || element.getAttribute('aria-required') === 'true' ? 'yes' : 'no', + disabled: element.getAttribute('data-disabled') === 'true' || element.getAttribute('aria-disabled') === 'true' ? 'yes' : 'no', + class: element.className || '', + label: label, + data_attributes: extractDataAttributes(element), + _selectors: selectors, + _extractId: extractId + }); + }); + + // Extract radio buttons (native and jQWidgets) + let radioIndex = 0; + + // Native radio buttons + document.querySelectorAll('input[type="radio"]').forEach(el => { + const radio = el as HTMLInputElement; + if (radio.offsetHeight > 0 || radio.offsetWidth > 0) { + const { selectors, extractId } = generateSelectors(radio, 'radio', radioIndex++); + elements.radios.push({ + type: 'radio', + name: radio.name || '', + id: radio.id || '', + value: radio.value || '', + checked: radio.checked ? 'yes' : 'no', + required: radio.required ? 'yes' : 'no', + disabled: radio.disabled ? 'yes' : 'no', + class: radio.className || '', + label: findAssociatedLabel(radio), + data_attributes: extractDataAttributes(radio), + _selectors: selectors, + _extractId: extractId + }); + } + }); + + // jQWidgets custom radio buttons + document.querySelectorAll('[xtype="radio"], .jqx-radiobutton, [role="radio"]:not(input)').forEach(el => { + const element = el as HTMLElement; + + // Skip if it's a native radio + if (element.tagName.toLowerCase() === 'input') return; + + // Skip if invisible + if (element.offsetHeight === 0 && element.offsetWidth === 0) return; + + const isChecked = element.getAttribute('aria-checked') === 'true' || + element.getAttribute('data-checked') === 'true' || + element.classList.contains('jqx-radiobutton-check-checked'); + + const { selectors, extractId } = generateSelectors(element, 'radio', radioIndex++); + elements.radios.push({ + type: 'radio-custom', + name: element.getAttribute('data-name') || element.getAttribute('name') || '', + id: element.id || '', + value: element.getAttribute('value') || element.getAttribute('data-value') || '', + checked: isChecked ? 'yes' : 'no', + required: element.getAttribute('data-required') === 'true' || element.getAttribute('aria-required') === 'true' ? 'yes' : 'no', + disabled: element.getAttribute('data-disabled') === 'true' || element.getAttribute('aria-disabled') === 'true' ? 'yes' : 'no', + class: element.className || '', + label: element.getAttribute('data-caption') || + element.getAttribute('aria-label') || + element.getAttribute('data-label') || + findAssociatedLabel(element), + data_attributes: extractDataAttributes(element), + _selectors: selectors, + _extractId: extractId + }); + }); + + // Extract switches (jQWidgets switchbutton and similar) + let switchIndex = 0; + document.querySelectorAll('[xtype="switch"], .jqx-switchbutton, [role="switch"]').forEach(el => { + const element = el as HTMLElement; + + // Skip if invisible + if (element.offsetHeight === 0 && element.offsetWidth === 0) return; + + const isOn = element.getAttribute('aria-checked') === 'true' || + element.getAttribute('data-checked') === 'true' || + element.classList.contains('jqx-switchbutton-checked'); + + const { selectors, extractId } = generateSelectors(element, 'switch', switchIndex++); + elements.switches.push({ + type: 'switch', + name: element.getAttribute('data-name') || element.getAttribute('name') || '', + id: element.id || '', + checked: isOn ? 'on' : 'off', + disabled: element.getAttribute('data-disabled') === 'true' || element.getAttribute('aria-disabled') === 'true' ? 'yes' : 'no', + class: element.className || '', + label: element.getAttribute('data-caption') || + element.getAttribute('aria-label') || + element.getAttribute('data-label') || + findAssociatedLabel(element), + data_attributes: extractDataAttributes(element), + _selectors: selectors, + _extractId: extractId + }); + }); + + // Extract sliders (jQWidgets slider and similar) + let sliderIndex = 0; + document.querySelectorAll('[xtype="slider"], .jqx-slider, [role="slider"]').forEach(el => { + const element = el as HTMLElement; + + // Skip if invisible + if (element.offsetHeight === 0 && element.offsetWidth === 0) return; + + const currentValue = element.getAttribute('aria-valuenow') || + element.getAttribute('data-value') || + element.getAttribute('value') || ''; + + const { selectors, extractId } = generateSelectors(element, 'slider', sliderIndex++); + elements.sliders.push({ + type: 'slider', + name: element.getAttribute('data-name') || element.getAttribute('name') || '', + id: element.id || '', + value: currentValue, + min: element.getAttribute('aria-valuemin') || element.getAttribute('data-min') || element.getAttribute('min') || '', + max: element.getAttribute('aria-valuemax') || element.getAttribute('data-max') || element.getAttribute('max') || '', + step: element.getAttribute('data-step') || element.getAttribute('step') || '', + disabled: element.getAttribute('data-disabled') === 'true' || element.getAttribute('aria-disabled') === 'true' ? 'yes' : 'no', + class: element.className || '', + label: element.getAttribute('data-caption') || + element.getAttribute('aria-label') || + element.getAttribute('data-label') || + findAssociatedLabel(element), + data_attributes: extractDataAttributes(element), + _selectors: selectors, + _extractId: extractId + }); + }); + + // Extract selects (native and custom dropdowns) + let selectIndex = 0; + + // Native + + + 🎯 + 智能优先 (7级回退) + + + + + 🟢 + Extract ID (最可靠) + + + + + 🟢 + CSS by ID (#id) + + + + + 🟡 + XPath Short (优化) + + + + + 🟡 + XPath Full (完整) + + + + + 🟡 + CSS Combined (组合) + + + + + 🟡 + CSS by Class (.class) + + + + + 🟡 + CSS by Attribute ([attr]) + + + + + 🔴 + CSS nth-child (位置) + + + + + + {/* Highlight Statistics */} + {highlightStats && highlightStats.show && ( + 0 || highlightStats.unavailable > 0 ? "warning" : "success"} + sx={{ mb: 1, py: 0 }} + onClose={() => setHighlightStats(null)} + > + + 高亮结果： + 成功 {highlightStats.success}/{highlightStats.total} + {highlightStats.unavailable > 0 && ` | 无此选择器 ${highlightStats.unavailable}`} + {highlightStats.failed > 0 && ` | 定位失败 ${highlightStats.failed}`} + + + )} + + {/* Action Buttons */} + + + + + + + {/* Tabs */} = ({ data }) => { + + + + + + + + + + + + + + + + + + + + void; +} + +export const PlaybackControl: React.FC = ({ session, onStateChange }) => { + const [playbackState, setPlaybackState] = useState('idle'); + const [progress, setProgress] = useState(null); + const [speed, setSpeed] = useState(1); + const [isLoading, setIsLoading] = useState(false); + + // Format time as MM:SS + const formatTime = (ms: number): string => { + const totalSeconds = Math.floor(ms / 1000); + const minutes = Math.floor(totalSeconds / 60); + const seconds = totalSeconds % 60; + return `${minutes.toString().padStart(2, '0')}:${seconds.toString().padStart(2, '0')}`; + }; + + // Listen for progress updates + useEffect(() => { + const listener = (message: any) => { + if (message.action === 'viewerProgress' && message.data.type === 'playback') { + setPlaybackState(message.data.state); + setProgress(message.data.playbackProgress || null); + + if (onStateChange) { + onStateChange(message.data.state); + } + } + }; + + chrome.runtime.onMessage.addListener(listener); + return () => { + chrome.runtime.onMessage.removeListener(listener); + }; + }, [onStateChange]); + + // Handle play + const handlePlay = async () => { + if (!session) { + alert('未加载会话'); + return; + } + + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + // First load the session + await chrome.tabs.sendMessage(tabs[0].id, { + action: 'startPlayback', + session, + speed + }); + } + } catch (error) { + console.error('Failed to start playback:', error); + alert('⚠️ 请先刷新页面！\n\n扩展需要在此页面重新加载。\n按 F5 或 Ctrl+R 刷新页面。'); + } finally { + setIsLoading(false); + } + }; + + // Handle pause + const handlePause = async () => { + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + await chrome.tabs.sendMessage(tabs[0].id, { action: 'pausePlayback' }); + } + } catch (error) { + console.error('Failed to pause playback:', error); + } finally { + setIsLoading(false); + } + }; + + // Handle stop + const handleStop = async () => { + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + await chrome.tabs.sendMessage(tabs[0].id, { action: 'stopPlayback' }); + } + } catch (error) { + console.error('Failed to stop playback:', error); + } finally { + setIsLoading(false); + } + }; + + // Handle seek forward/backward + const handleSeek = async (offset: number) => { + if (!progress) return; + + const newTime = Math.max(0, Math.min(progress.totalTime, progress.currentTime + offset)); + + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + await chrome.tabs.sendMessage(tabs[0].id, { + action: 'seekPlayback', + time: newTime + }); + } + } catch (error) { + console.error('Failed to seek:', error); + } + }; + + // Handle speed change + const handleSpeedChange = (newSpeed: number) => { + setSpeed(newSpeed); + // Speed will be applied on next play + }; + + // Calculate progress percentage + const progressPercent = progress && progress.totalTime > 0 + ? (progress.currentTime / progress.totalTime) * 100 + : 0; + + // Calculate success rate + const successRate = progress && progress.results.length > 0 + ? (progress.results.filter(r => r.success).length / progress.results.length) * 100 + : 0; + + if (!session) { + return ( + + + 未加载回放会话。请先录制或导入一个会话。 + + + ); + } + + return ( + + + {/* Title */} + + ▶️ 回放控制 + + + {/* Status Display */} + + + {progress && ( + <> + + + {progress.results.length > 0 && ( + + )} + + )} + + + {/* Progress Bar */} + {progress && progress.totalTime > 0 && ( + + + + )} + + {/* Speed Control */} + + 速度 + + + + {/* Control Buttons */} + + {(playbackState === 'idle' || playbackState === 'stopped') && ( + + )} + + {playbackState === 'playing' && ( + <> + + + + + )} + + {playbackState === 'paused' && ( + <> + + + + )} + + {(playbackState === 'playing' || playbackState === 'paused') && ( + + )} + + + {/* Stats Display */} + {progress && progress.results.length > 0 && ( + + + 已执行：{progress.results.length} / {progress.totalActions} + + + 成功：{progress.results.filter(r => r.success).length} | + 失败：{progress.results.filter(r => !r.success).length} + + + )} + + {/* Help Text */} + {playbackState === 'idle' && ( + + 点击"播放"以 {speed}x 速度开始重放录制的会话。 + + )} + + + ); +}; + diff --git a/src/sidepanel/components/RecordingControl.tsx b/src/sidepanel/components/RecordingControl.tsx new file mode 100644 index 0000000..45696c2 --- /dev/null +++ b/src/sidepanel/components/RecordingControl.tsx @@ -0,0 +1,316 @@ +import React, { useState, useEffect } from 'react'; +import { + Box, + Button, + Typography, + Paper, + Stack, + Chip +} from '@mui/material'; +import { + RadioButtonChecked, + Pause, + PlayArrow, + Stop +} from '@mui/icons-material'; +import type { RecordingState } from '../types'; + +interface RecordingControlProps { + onStateChange?: (state: RecordingState) => void; +} + +export const RecordingControl: React.FC = ({ onStateChange }) => { + const [recordingState, setRecordingState] = useState('idle'); + const [actionCount, setActionCount] = useState(0); + const [duration, setDuration] = useState(0); + const [isLoading, setIsLoading] = useState(false); + + // Format duration as MM:SS + const formatDuration = (ms: number): string => { + const totalSeconds = Math.floor(ms / 1000); + const minutes = Math.floor(totalSeconds / 60); + const seconds = totalSeconds % 60; + return `${minutes.toString().padStart(2, '0')}:${seconds.toString().padStart(2, '0')}`; + }; + + // Listen for progress updates from content script + useEffect(() => { + const listener = (message: any) => { + if (message.action === 'viewerProgress' && message.data.type === 'recording') { + setRecordingState(message.data.state); + setActionCount(message.data.actionCount || 0); + setDuration(message.data.duration || 0); + + if (onStateChange) { + onStateChange(message.data.state); + } + } + }; + + chrome.runtime.onMessage.addListener(listener); + return () => { + chrome.runtime.onMessage.removeListener(listener); + }; + }, [onStateChange]); + + // Poll for state updates + useEffect(() => { + const interval = setInterval(() => { + if (recordingState === 'recording') { + // Request state update + chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { + if (tabs[0]?.id) { + chrome.tabs.sendMessage(tabs[0].id, { action: 'getRecordingState' }) + .catch(() => { + // Content script may not be loaded + }); + } + }); + } + }, 500); + + return () => clearInterval(interval); + }, [recordingState]); + + // Handle start recording + const handleStart = async () => { + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + const response = await chrome.tabs.sendMessage(tabs[0].id, { + action: 'startRecording' + }); + + if (response.success) { + setRecordingState('recording'); + setActionCount(0); + setDuration(0); + + if (onStateChange) { + onStateChange('recording'); + } + } + } + } catch (error) { + console.error('Failed to start recording:', error); + alert('⚠️ 请先刷新页面！\n\n扩展需要在此页面重新加载。\n按 F5 或 Ctrl+R 刷新页面。'); + } finally { + setIsLoading(false); + } + }; + + // Handle pause recording + const handlePause = async () => { + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + const response = await chrome.tabs.sendMessage(tabs[0].id, { + action: 'pauseRecording' + }); + + if (response.success) { + setRecordingState('paused'); + + if (onStateChange) { + onStateChange('paused'); + } + } + } + } catch (error) { + console.error('Failed to pause recording:', error); + } finally { + setIsLoading(false); + } + }; + + // Handle resume recording + const handleResume = async () => { + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + const response = await chrome.tabs.sendMessage(tabs[0].id, { + action: 'resumeRecording' + }); + + if (response.success) { + setRecordingState('recording'); + + if (onStateChange) { + onStateChange('recording'); + } + } + } + } catch (error) { + console.error('Failed to resume recording:', error); + } finally { + setIsLoading(false); + } + }; + + // Handle stop recording + const handleStop = async () => { + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + const response = await chrome.tabs.sendMessage(tabs[0].id, { + action: 'stopRecording' + }); + + if (response.success) { + setRecordingState('stopped'); + + if (onStateChange) { + onStateChange('stopped'); + } + + // Reset to idle after a moment + setTimeout(() => { + setRecordingState('idle'); + if (onStateChange) { + onStateChange('idle'); + } + }, 1000); + } + } + } catch (error) { + console.error('Failed to stop recording:', error); + } finally { + setIsLoading(false); + } + }; + + return ( + + + {/* Title */} + + 🎬 录制控制 + + + {/* Status Display */} + + + {(recordingState === 'recording' || recordingState === 'paused') && ( + <> + + + + )} + + + {/* Control Buttons */} + + {recordingState === 'idle' && ( + + )} + + {recordingState === 'recording' && ( + <> + + + + )} + + {recordingState === 'paused' && ( + <> + + + + )} + + {recordingState === 'stopped' && ( + + ✅ 录制已保存！您现在可以导出了。 + + )} + + + {/* Help Text */} + {recordingState === 'idle' && ( + + 点击"开始录制"以开始跟踪此页面上的所有用户交互。 + + )} + + + ); +}; + diff --git a/src/sidepanel/components/RecordingTimeline.tsx b/src/sidepanel/components/RecordingTimeline.tsx new file mode 100644 index 0000000..de5a93d --- /dev/null +++ b/src/sidepanel/components/RecordingTimeline.tsx @@ -0,0 +1,267 @@ +import React from 'react'; +import { + Box, + Paper, + Typography, + Slider, + Tooltip +} from '@mui/material'; +import type { UserAction, RecordingSession } from '../types'; + +interface RecordingTimelineProps { + session: RecordingSession | null; + currentTime?: number; + onSeek?: (time: number) => void; + onActionClick?: (action: UserAction) => void; +} + +export const RecordingTimeline: React.FC = ({ + session, + currentTime = 0, + onSeek, + onActionClick +}) => { + if (!session || session.actions.length === 0) { + return null; + } + + const totalDuration = session.duration || 0; + + // Format time as MM:SS.mmm + const formatTime = (ms: number): string => { + const totalSeconds = Math.floor(ms / 1000); + const minutes = Math.floor(totalSeconds / 60); + const seconds = totalSeconds % 60; + const milliseconds = ms % 1000; + return `${minutes.toString().padStart(2, '0')}:${seconds.toString().padStart(2, '0')}.${Math.floor(milliseconds / 100)}`; + }; + + // Get color for action type + const getActionColor = (type: string): string => { + switch (type) { + case 'click': + case 'dblclick': + return '#2196f3'; // Blue + case 'scroll': + case 'scrollElement': + return '#4caf50'; // Green + case 'input': + case 'change': + return '#ff9800'; // Orange + case 'keydown': + case 'keyup': + return '#9c27b0'; // Purple + case 'navigation': + return '#f44336'; // Red + default: + return '#757575'; // Grey + } + }; + + // Get action type icon + const getActionIcon = (type: string): string => { + switch (type) { + case 'click': + case 'dblclick': + return '🖱️'; + case 'scroll': + case 'scrollElement': + return '📜'; + case 'input': + case 'change': + return '⌨️'; + case 'keydown': + case 'keyup': + return '🔤'; + case 'navigation': + return '🔗'; + case 'submit': + return '📤'; + case 'focus': + return '🎯'; + case 'drag start': + case 'dragend': + return '🔀'; + default: + return '•'; + } + }; + + // Handle slider change + const handleSliderChange = (_event: Event, value: number | number[]) => { + const time = Array.isArray(value) ? value[0] : value; + if (onSeek) { + onSeek(time); + } + }; + + // Create marks for significant actions + const marks = session.actions + .filter((_, index) => index % Math.max(1, Math.floor(session.actions.length / 20)) === 0) + .map(action => ({ + value: action.timestamp, + label: '' + })); + + return ( + + {/* Title */} + + 📊 时间轴（{session.actions.length} 个操作） + + + {/* Timeline Slider */} + + + + + {/* Time Display */} + + + {formatTime(currentTime)} + + + {formatTime(totalDuration)} + + + + {/* Action Dots Visualization */} + + {session.actions.map((action, index) => { + const position = (action.timestamp / totalDuration) * 100; + const color = getActionColor(action.type); + + return ( + + + {getActionIcon(action.type)} {action.type} + + + {formatTime(action.timestamp)} + + {action.target?.text && ( + + {action.target.text} + + )} + + } + arrow + > + { + if (onActionClick) { + onActionClick(action); + } + if (onSeek) { + onSeek(action.timestamp); + } + }} + sx={{ + position: 'absolute', + left: `${position}%`, + top: '50%', + transform: 'translate(-50%, -50%)', + width: 8, + height: 8, + borderRadius: '50%', + backgroundColor: color, + cursor: 'pointer', + transition: 'all 0.2s', + '&:hover': { + width: 12, + height: 12, + boxShadow: `0 0 8px ${color}` + } + }} + /> + + ); + })} + + {/* Current Time Indicator */} + + + + {/* Legend */} + + {[ + { type: 'click', label: '点击' }, + { type: 'scroll', label: '滚动' }, + { type: 'input', label: '输入' }, + { type: 'keydown', label: '键盘' }, + { type: 'navigation', label: '导航' } + ].map(({ type, label }) => ( + + + + {getActionIcon(type)} {label} + + + ))} + + + ); +}; + diff --git a/src/sidepanel/components/SelectorDetails.tsx b/src/sidepanel/components/SelectorDetails.tsx index 1dbfdac..1e5d566 100644 --- a/src/sidepanel/components/SelectorDetails.tsx +++ b/src/sidepanel/components/SelectorDetails.tsx @@ -45,18 +45,19 @@ export const SelectorDetails: React.FC = ({ ]; return ( - - - + + + 可用选择器（点击选择器以高亮元素） - + {selectorItems.map((item, idx) => ( = ({ '&:hover': { backgroundColor: '#e3f2fd', borderColor: '#2196f3', - transform: 'translateX(4px)' + transform: 'translateX(2px)' } }} onClick={(e) => onSelectorClick(item.label, item.value, extractId, e)} @@ -82,19 +83,19 @@ export const SelectorDetails: React.FC = ({ > + {item.reliability} - + {item.label} @@ -105,9 +106,10 @@ export const SelectorDetails: React.FC = ({ variant="body2" sx={{ fontFamily: 'monospace', - fontSize: '0.75rem', + fontSize: '0.7rem', wordBreak: 'break-all', - mt: 0.5 + mt: 0.25, + lineHeight: 1.3 }} > {item.value} diff --git a/src/sidepanel/components/SessionManager.tsx b/src/sidepanel/components/SessionManager.tsx new file mode 100644 index 0000000..86afbe7 --- /dev/null +++ b/src/sidepanel/components/SessionManager.tsx @@ -0,0 +1,313 @@ +import React, { useState, useRef } from 'react'; +import { + Box, + Button, + Paper, + Typography, + Stack, + Chip, + Dialog, + DialogTitle, + DialogContent, + DialogActions +} from '@mui/material'; +import { + Download, + Upload, + Delete, + BarChart +} from '@mui/icons-material'; +import type { RecordingSession } from '../types'; + +interface SessionManagerProps { + session: RecordingSession | null; + onSessionLoaded?: (session: RecordingSession) => void; + onSessionCleared?: () => void; +} + +export const SessionManager: React.FC = ({ + session, + onSessionLoaded, + onSessionCleared +}) => { + const [isLoading, setIsLoading] = useState(false); + const [showStats, setShowStats] = useState(false); + const fileInputRef = useRef(null); + + // Format duration + const formatDuration = (ms: number): string => { + const totalSeconds = Math.floor(ms / 1000); + const minutes = Math.floor(totalSeconds / 60); + const seconds = totalSeconds % 60; + return `${minutes}m ${seconds}s`; + }; + + // Format date + const formatDate = (timestamp: number): string => { + return new Date(timestamp).toLocaleString(); + }; + + // Handle export + const handleExport = async () => { + if (!session) { + alert('没有可导出的会话'); + return; + } + + try { + const json = JSON.stringify(session, null, 2); + const blob = new Blob([json], { type: 'application/json' }); + const url = URL.createObjectURL(blob); + + const a = document.createElement('a'); + a.href = url; + a.download = `recording-${session.id}-${new Date().toISOString().replace(/[:.]/g, '-')}.json`; + document.body.appendChild(a); + a.click(); + document.body.removeChild(a); + + URL.revokeObjectURL(url); + + alert('✅ 会话导出成功！'); + } catch (error) { + console.error('Failed to export session:', error); + alert('❌ 导出会话失败'); + } + }; + + // Handle import + const handleImport = () => { + fileInputRef.current?.click(); + }; + + // Handle file selection + const handleFileChange = async (event: React.ChangeEvent) => { + const file = event.target.files?.[0]; + if (!file) return; + + setIsLoading(true); + try { + const text = await file.text(); + const importedSession = JSON.parse(text) as RecordingSession; + + // Validate session + if (!importedSession.id || !importedSession.actions || !Array.isArray(importedSession.actions)) { + throw new Error('Invalid session format'); + } + + if (onSessionLoaded) { + onSessionLoaded(importedSession); + } + + alert(`✅ 会话导入成功！已加载 ${importedSession.actions.length} 个操作。`); + } catch (error) { + console.error('Failed to import session:', error); + alert('❌ 导入会话失败。请检查文件格式。'); + } finally { + setIsLoading(false); + // Reset file input + if (fileInputRef.current) { + fileInputRef.current.value = ''; + } + } + }; + + // Handle delete + const handleDelete = async () => { + if (!confirm('确定要删除当前会话吗？此操作无法撤销。')) { + return; + } + + setIsLoading(true); + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + await chrome.tabs.sendMessage(tabs[0].id, { action: 'clearRecording' }); + } + + if (onSessionCleared) { + onSessionCleared(); + } + + alert('✅ 会话清除成功！'); + } catch (error) { + console.error('Failed to clear session:', error); + alert('❌ 清除会话失败'); + } finally { + setIsLoading(false); + } + }; + + // Get action type statistics + const getActionStats = () => { + if (!session) return {}; + + const stats: Record = {}; + session.actions.forEach(action => { + stats[action.type] = (stats[action.type] || 0) + 1; + }); + + return stats; + }; + + const actionStats = session ? getActionStats() : {}; + + return ( + <> + + + {/* Title */} + + 💾 会话管理 + + + {/* Session Info */} + {session && ( + + + 会话 ID： {session.id} + + + URL： {session.url} + + + 标题： {session.title} + + + 时长： {formatDuration(session.duration)} + + + 操作数： {session.actions.length} + + + 录制时间： {formatDate(session.startTime)} + + + )} + + {/* No Session Message */} + {!session && ( + + 未加载会话。录制新会话或导入现有会话。 + + )} + + {/* Action Buttons */} + + + + + + {session && ( + + + + + )} + + {/* Hidden file input */} + + + + + {/* Stats Dialog */} + setShowStats(false)} + maxWidth="sm" + fullWidth + > + 会话统计 + + + + + 操作类型： + + + {Object.entries(actionStats) + .sort(([, a], [, b]) => b - a) + .map(([type, count]) => ( + + ))} + + + + {session && ( + + + 元数据： + + + 用户代理：{session.metadata.userAgent} + + + 屏幕分辨率：{session.metadata.screenResolution} + + + 时区：{session.metadata.timezone} + + + )} + + + + + + + + ); +}; + diff --git a/src/sidepanel/components/ViewerTab.tsx b/src/sidepanel/components/ViewerTab.tsx new file mode 100644 index 0000000..8100cae --- /dev/null +++ b/src/sidepanel/components/ViewerTab.tsx @@ -0,0 +1,129 @@ +import React, { useState, useEffect } from 'react'; +import { Box, Stack } from '@mui/material'; +import { RecordingControl } from './RecordingControl'; +import { RecordingTimeline } from './RecordingTimeline'; +import { ActionsList } from './ActionsList'; +import { PlaybackControl } from './PlaybackControl'; +import { SessionManager } from './SessionManager'; +import type { RecordingSession, RecordingState, UserAction } from '../types'; + +export const ViewerTab: React.FC = () => { + const [currentSession, setCurrentSession] = useState(null); + const [recordingState, setRecordingState] = useState('idle'); + const [currentPlaybackTime, setCurrentPlaybackTime] = useState(0); + + // Load current session on mount + useEffect(() => { + loadCurrentSession(); + }, []); + + // Listen for recording state changes + useEffect(() => { + if (recordingState === 'stopped') { + // Reload session after recording stops + setTimeout(() => { + loadCurrentSession(); + }, 500); + } + }, [recordingState]); + + // Load current session from storage + const loadCurrentSession = async () => { + try { + const tabs = await chrome.tabs.query({ active: true, currentWindow: true }); + if (tabs[0]?.id) { + const response = await chrome.tabs.sendMessage(tabs[0].id, { + action: 'getRecordingState' + }); + + if (response?.success && response?.data?.session) { + setCurrentSession(response.data.session); + } + } + } catch (error) { + // Content script may not be loaded or no session available + console.log('No session loaded:', error); + } + }; + + // Handle session loaded (from import) + const handleSessionLoaded = (session: RecordingSession) => { + setCurrentSession(session); + }; + + // Handle session cleared + const handleSessionCleared = () => { + setCurrentSession(null); + }; + + // Handle action click from list + const handleActionClick = (action: UserAction) => { + // Highlight the element on the page + if (action.target?.selectors) { + chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { + if (tabs[0]?.id) { + chrome.tabs.sendMessage(tabs[0].id, { + action: 'highlightElement', + selectors: action.target!.selectors, + extractId: action.target!.extractId + }); + } + }); + } + }; + + // Handle timeline seek + const handleTimelineSeek = (time: number) => { + setCurrentPlaybackTime(time); + // Send seek command to player + chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { + if (tabs[0]?.id) { + chrome.tabs.sendMessage(tabs[0].id, { + action: 'seekPlayback', + time + }); + } + }); + }; + + return ( + + + {/* Recording Control */} + + + {/* Session Manager */} + + + {/* Show timeline and playback controls only when session is available */} + {currentSession && ( + <> + {/* Playback Control */} + + + {/* Timeline */} + + + {/* Actions List */} + + + )} + + + ); +}; + diff --git a/src/sidepanel/components/tabs/ButtonsTab.tsx b/src/sidepanel/components/tabs/ButtonsTab.tsx index 4c4e9b9..ed63175 100644 --- a/src/sidepanel/components/tabs/ButtonsTab.tsx +++ b/src/sidepanel/components/tabs/ButtonsTab.tsx @@ -31,7 +31,7 @@ export const ButtonsTab: React.FC = ({ }) => { return ( <> - + diff --git a/src/sidepanel/components/tabs/CheckboxesTab.tsx b/src/sidepanel/components/tabs/CheckboxesTab.tsx new file mode 100644 index 0000000..bdc30bc --- /dev/null +++ b/src/sidepanel/components/tabs/CheckboxesTab.tsx @@ -0,0 +1,91 @@ +import React from 'react'; +import { + Table, + TableBody, + TableCell, + TableContainer, + TableHead, + TableRow, + Collapse, + IconButton, + Typography +} from '@mui/material'; +import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown'; +import KeyboardArrowUpIcon from '@mui/icons-material/KeyboardArrowUp'; +import type { CheckboxElement, ElementSelectors } from '../../types'; + +interface CheckboxesTabProps { + elements: CheckboxElement[]; + expandedRow: string | null; + searchQuery: string; + handleRowClick: (rowId: string) => void; + renderSelectorDetails: (selectors?: ElementSelectors, extractId?: string) => React.ReactNode; +} + +export const CheckboxesTab: React.FC = ({ + elements, + expandedRow, + searchQuery, + handleRowClick, + renderSelectorDetails +}) => { + return ( + <> + +
+ + + + 标签 + Name + ID + Value + 选中 + 必填 + + + + {elements.map((checkbox, idx) => { + const rowId = `checkbox-${idx}`; + const isExpanded = expandedRow === rowId; + return ( + + handleRowClick(rowId)} + sx={{ cursor: 'pointer', '&:hover': { backgroundColor: '#f5f5f5' } }} + > + + + {isExpanded ? : } + + + {checkbox.label || '-'} + {checkbox.name || '-'} + {checkbox.id || '-'} + {checkbox.value || '-'} + {checkbox.checked === 'yes' ? '☑️' : '☐'} + {checkbox.required === 'yes' ? '✅' : '-'} + + + + + {renderSelectorDetails(checkbox._selectors, checkbox._extractId)} + + + + + ); + })} + +
+ + {elements.length === 0 && ( + + {searchQuery ? '未找到匹配的复选框元素' : '未找到复选框元素'} + + )} + + ); +}; + diff --git a/src/sidepanel/components/tabs/CustomTab.tsx b/src/sidepanel/components/tabs/CustomTab.tsx index fee2e5e..eb94b2a 100644 --- a/src/sidepanel/components/tabs/CustomTab.tsx +++ b/src/sidepanel/components/tabs/CustomTab.tsx @@ -31,7 +31,7 @@ export const CustomTab: React.FC = ({ }) => { return ( <> - + diff --git a/src/sidepanel/components/tabs/DatatablesTab.tsx b/src/sidepanel/components/tabs/DatatablesTab.tsx new file mode 100644 index 0000000..f065cef --- /dev/null +++ b/src/sidepanel/components/tabs/DatatablesTab.tsx @@ -0,0 +1,243 @@ +import React from 'react'; +import { + Table, + TableBody, + TableCell, + TableContainer, + TableHead, + TableRow, + Collapse, + IconButton, + Typography, + Chip, + Box +} from '@mui/material'; +import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown'; +import KeyboardArrowUpIcon from '@mui/icons-material/KeyboardArrowUp'; +import TableChartIcon from '@mui/icons-material/TableChart'; +import CheckCircleIcon from '@mui/icons-material/CheckCircle'; +import CancelIcon from '@mui/icons-material/Cancel'; +import type { DatatableElement, ElementSelectors } from '../../types'; + +interface DatatablesTabProps { + elements: DatatableElement[]; + expandedRow: string | null; + searchQuery: string; + handleRowClick: (rowId: string) => void; + renderSelectorDetails: (selectors?: ElementSelectors, extractId?: string) => React.ReactNode; +} + +export const DatatablesTab: React.FC = ({ + elements, + expandedRow, + searchQuery, + handleRowClick, + renderSelectorDetails +}) => { + const getFeatureIcon = (hasFeature: boolean) => { + return hasFeature ? ( + + ) : ( + + ); + }; + + return ( + <> + +
+ + + + 表格类型 + ID + 结构 + 列名 + 功能 + 分页信息 + + + + {elements.map((datatable, idx) => { + const rowId = `datatable-${idx}`; + const isExpanded = expandedRow === rowId; + const structure = `${datatable.rowCount}行 × ${datatable.columnCount}列`; + const paginationInfo = datatable.hasPagination + ? `${datatable.currentPage}/${datatable.totalPages}页 (共${datatable.totalRecords}条)` + : '-'; + + return ( + + handleRowClick(rowId)} + sx={{ cursor: 'pointer', '&:hover': { backgroundColor: '#f5f5f5' } }} + > + + + {isExpanded ? : } + + + + + + + + + + + {datatable.id || '-'} + + + + + {structure} + + + + + {datatable.columns || '-'} + + + + + + {getFeatureIcon(datatable.hasPagination)} + + + {getFeatureIcon(datatable.hasSorting)} + + + {getFeatureIcon(datatable.hasFiltering)} + + + + + + {paginationInfo} + + + + + + + + {/* Table Details */} + + 📊 表格详情 + + + + 标签: {datatable.tag} + + + Class: {datatable.class || '-'} + + + 列名: {datatable.columns || '-'} + + + 总记录数: {datatable.totalRecords} + + + + {/* Features */} + + ⚙️ 功能特性 + + + + 分页: {datatable.hasPagination ? '✓ 支持' : '✗ 不支持'} + + + 排序: {datatable.hasSorting ? '✓ 支持' : '✗ 不支持'} + + + 筛选: {datatable.hasFiltering ? '✓ 支持' : '✗ 不支持'} + + + + {/* EMAP Metadata (if exists) */} + {datatable.type === 'emap-datatable' && (datatable.emapAction || datatable.emapPagePath) && ( + <> + + 🏷️ EMAP 元数据 + + + {datatable.emapAction && ( + + Action: {datatable.emapAction} + + )} + {datatable.emapPagePath && ( + + PagePath: {datatable.emapPagePath} + + )} + {datatable.emapAppName && ( + + AppName: {datatable.emapAppName} + + )} + + + )} + + {/* Row Sample Preview */} + {datatable.rowSample && ( + <> + + 📋 数据预览（前3行） + + + 
+                                  {datatable.rowSample}
+
+ + + )} + + {/* Selectors */} + {renderSelectorDetails(datatable._selectors, datatable._extractId)} + + + + + + ); + })} + +
+ + {elements.length === 0 && ( + + {searchQuery ? '未找到匹配的数据表格' : '未找到数据表格'} + + )} + + ); +}; + diff --git a/src/sidepanel/components/tabs/FormsTab.tsx b/src/sidepanel/components/tabs/FormsTab.tsx index 8912545..0d40836 100644 --- a/src/sidepanel/components/tabs/FormsTab.tsx +++ b/src/sidepanel/components/tabs/FormsTab.tsx @@ -31,7 +31,7 @@ export const FormsTab: React.FC = ({ }) => { return ( <> - + diff --git a/src/sidepanel/components/tabs/InputsTab.tsx b/src/sidepanel/components/tabs/InputsTab.tsx index 2d62c94..429f0cd 100644 --- a/src/sidepanel/components/tabs/InputsTab.tsx +++ b/src/sidepanel/components/tabs/InputsTab.tsx @@ -31,7 +31,7 @@ export const InputsTab: React.FC = ({ }) => { return ( <> - +
diff --git a/src/sidepanel/components/tabs/LinksTab.tsx b/src/sidepanel/components/tabs/LinksTab.tsx index 0f33375..032ca09 100644 --- a/src/sidepanel/components/tabs/LinksTab.tsx +++ b/src/sidepanel/components/tabs/LinksTab.tsx @@ -31,7 +31,7 @@ export const LinksTab: React.FC = ({ }) => { return ( <> - +
diff --git a/src/sidepanel/components/tabs/RadiosTab.tsx b/src/sidepanel/components/tabs/RadiosTab.tsx new file mode 100644 index 0000000..afeaf10 --- /dev/null +++ b/src/sidepanel/components/tabs/RadiosTab.tsx @@ -0,0 +1,91 @@ +import React from 'react'; +import { + Table, + TableBody, + TableCell, + TableContainer, + TableHead, + TableRow, + Collapse, + IconButton, + Typography +} from '@mui/material'; +import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown'; +import KeyboardArrowUpIcon from '@mui/icons-material/KeyboardArrowUp'; +import type { RadioElement, ElementSelectors } from '../../types'; + +interface RadiosTabProps { + elements: RadioElement[]; + expandedRow: string | null; + searchQuery: string; + handleRowClick: (rowId: string) => void; + renderSelectorDetails: (selectors?: ElementSelectors, extractId?: string) => React.ReactNode; +} + +export const RadiosTab: React.FC = ({ + elements, + expandedRow, + searchQuery, + handleRowClick, + renderSelectorDetails +}) => { + return ( + <> + +
+ + + + 标签 + Name + ID + Value + 选中 + 必填 + + + + {elements.map((radio, idx) => { + const rowId = `radio-${idx}`; + const isExpanded = expandedRow === rowId; + return ( + + handleRowClick(rowId)} + sx={{ cursor: 'pointer', '&:hover': { backgroundColor: '#f5f5f5' } }} + > + + + {isExpanded ? : } + + + {radio.label || '-'} + {radio.name || '-'} + {radio.id || '-'} + {radio.value || '-'} + {radio.checked === 'yes' ? '🔘' : '⚪'} + {radio.required === 'yes' ? '✅' : '-'} + + + + + {renderSelectorDetails(radio._selectors, radio._extractId)} + + + + + ); + })} + +
+ + {elements.length === 0 && ( + + {searchQuery ? '未找到匹配的单选框元素' : '未找到单选框元素'} + + )} + + ); +}; + diff --git a/src/sidepanel/components/tabs/SelectsTab.tsx b/src/sidepanel/components/tabs/SelectsTab.tsx index f0f551d..0c11b30 100644 --- a/src/sidepanel/components/tabs/SelectsTab.tsx +++ b/src/sidepanel/components/tabs/SelectsTab.tsx @@ -31,7 +31,7 @@ export const SelectsTab: React.FC = ({ }) => { return ( <> - + diff --git a/src/sidepanel/components/tabs/SlidersTab.tsx b/src/sidepanel/components/tabs/SlidersTab.tsx new file mode 100644 index 0000000..7323f55 --- /dev/null +++ b/src/sidepanel/components/tabs/SlidersTab.tsx @@ -0,0 +1,92 @@ +import React from 'react'; +import { + Table, + TableBody, + TableCell, + TableContainer, + TableHead, + TableRow, + Collapse, + IconButton, + Typography +} from '@mui/material'; +import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown'; +import KeyboardArrowUpIcon from '@mui/icons-material/KeyboardArrowUp'; +import type { SliderElement, ElementSelectors } from '../../types'; + +interface SlidersTabProps { + elements: SliderElement[]; + expandedRow: string | null; + searchQuery: string; + handleRowClick: (rowId: string) => void; + renderSelectorDetails: (selectors?: ElementSelectors, extractId?: string) => React.ReactNode; +} + +export const SlidersTab: React.FC = ({ + elements, + expandedRow, + searchQuery, + handleRowClick, + renderSelectorDetails +}) => { + return ( + <> + +
+ + + + 标签 + Name + ID + 当前值 + 范围 + 步进 + + + + {elements.map((slider, idx) => { + const rowId = `slider-${idx}`; + const isExpanded = expandedRow === rowId; + const range = slider.min && slider.max ? `${slider.min} ~ ${slider.max}` : '-'; + return ( + + handleRowClick(rowId)} + sx={{ cursor: 'pointer', '&:hover': { backgroundColor: '#f5f5f5' } }} + > + + + {isExpanded ? : } + + + {slider.label || '-'} + {slider.name || '-'} + {slider.id || '-'} + {slider.value || '-'} + {range} + {slider.step || '-'} + + + + + {renderSelectorDetails(slider._selectors, slider._extractId)} + + + + + ); + })} + +
+ + {elements.length === 0 && ( + + {searchQuery ? '未找到匹配的滑块元素' : '未找到滑块元素'} + + )} + + ); +}; + diff --git a/src/sidepanel/components/tabs/SwitchesTab.tsx b/src/sidepanel/components/tabs/SwitchesTab.tsx new file mode 100644 index 0000000..cbe0025 --- /dev/null +++ b/src/sidepanel/components/tabs/SwitchesTab.tsx @@ -0,0 +1,89 @@ +import React from 'react'; +import { + Table, + TableBody, + TableCell, + TableContainer, + TableHead, + TableRow, + Collapse, + IconButton, + Typography +} from '@mui/material'; +import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown'; +import KeyboardArrowUpIcon from '@mui/icons-material/KeyboardArrowUp'; +import type { SwitchElement, ElementSelectors } from '../../types'; + +interface SwitchesTabProps { + elements: SwitchElement[]; + expandedRow: string | null; + searchQuery: string; + handleRowClick: (rowId: string) => void; + renderSelectorDetails: (selectors?: ElementSelectors, extractId?: string) => React.ReactNode; +} + +export const SwitchesTab: React.FC = ({ + elements, + expandedRow, + searchQuery, + handleRowClick, + renderSelectorDetails +}) => { + return ( + <> + + + + + + 标签 + Name + ID + 状态 + 禁用 + + + + {elements.map((switchElem, idx) => { + const rowId = `switch-${idx}`; + const isExpanded = expandedRow === rowId; + return ( + + handleRowClick(rowId)} + sx={{ cursor: 'pointer', '&:hover': { backgroundColor: '#f5f5f5' } }} + > + + + {isExpanded ? : } + + + {switchElem.label || '-'} + {switchElem.name || '-'} + {switchElem.id || '-'} + {switchElem.checked === 'on' ? '🟢 ON' : '⚪ OFF'} + {switchElem.disabled === 'yes' ? '🚫' : '-'} + + + + + {renderSelectorDetails(switchElem._selectors, switchElem._extractId)} + + + + + ); + })} + +
+ + {elements.length === 0 && ( + + {searchQuery ? '未找到匹配的开关元素' : '未找到开关元素'} + + )} + + ); +}; + diff --git a/src/sidepanel/components/tabs/TextareasTab.tsx b/src/sidepanel/components/tabs/TextareasTab.tsx index 4ee6545..155387f 100644 --- a/src/sidepanel/components/tabs/TextareasTab.tsx +++ b/src/sidepanel/components/tabs/TextareasTab.tsx @@ -31,7 +31,7 @@ export const TextareasTab: React.FC = ({ }) => { return ( <> - + diff --git a/src/sidepanel/services/elementExtractor.ts b/src/sidepanel/services/elementExtractor.ts index 6872c98..940bfaf 100644 --- a/src/sidepanel/services/elementExtractor.ts +++ b/src/sidepanel/services/elementExtractor.ts @@ -14,6 +14,11 @@ export function extractInteractiveElementsInline(): ElementData { inputs: [], selects: [], textareas: [], + checkboxes: [], + radios: [], + switches: [], + sliders: [], + datatables: [], custom: [] }; @@ -389,6 +394,11 @@ export function extractInteractiveElementsInline(): ElementData { inputs: elements.inputs.length, selects: elements.selects.length, textareas: elements.textareas.length, + checkboxes: elements.checkboxes.length, + radios: elements.radios.length, + switches: elements.switches.length, + sliders: elements.sliders.length, + datatables: elements.datatables.length, custom: elements.custom.length } }; @@ -408,6 +418,11 @@ export function extractInteractiveElementsInline(): ElementData { inputs: 0, selects: 0, textareas: 0, + checkboxes: 0, + radios: 0, + switches: 0, + sliders: 0, + datatables: 0, custom: 0 } }; diff --git a/src/sidepanel/types/index.ts b/src/sidepanel/types/index.ts index a1c0040..fcb3b7e 100644 --- a/src/sidepanel/types/index.ts +++ b/src/sidepanel/types/index.ts @@ -36,6 +36,25 @@ export interface SaveResult { export type ExportFormat = 'html' | 'xlsx' | 'csv' | 'json'; +// Selector type for batch highlighting +export type SelectorType = + | 'auto' // Smart fallback (default) + | 'extractId' // data-extract-id + | 'cssById' // #id + | 'xpathFull' // Full XPath + | 'xpathShort' // Optimized XPath + | 'cssCombined' // Combined CSS + | 'cssByClass' // Class-based + | 'cssByAttribute' // Attribute-based + | 'cssByNthChild'; // nth-child + +// Used selector information (actual selector used for highlighting) +export interface UsedSelectorInfo { + type: string; // 'cssById' | 'xpathShort' etc. + value: string; // Actual selector value + reliability: 'high' | 'medium' | 'low'; +} + // Selector information for element location export interface ElementSelectors { xpath: string; // Full XPath @@ -140,6 +159,65 @@ export interface TextareaElement { _extractId?: string; } +export interface CheckboxElement { + type: 'checkbox' | 'checkbox-custom'; + name: string; + id: string; + value: string; + checked: 'yes' | 'no'; + required: 'yes' | 'no'; + disabled: 'yes' | 'no'; + class: string; + label: string; + data_attributes: string; + _selectors?: ElementSelectors; + _extractId?: string; +} + +export interface RadioElement { + type: 'radio' | 'radio-custom'; + name: string; + id: string; + value: string; + checked: 'yes' | 'no'; + required: 'yes' | 'no'; + disabled: 'yes' | 'no'; + class: string; + label: string; + data_attributes: string; + _selectors?: ElementSelectors; + _extractId?: string; +} + +export interface SwitchElement { + type: 'switch'; + name: string; + id: string; + checked: 'on' | 'off'; + disabled: 'yes' | 'no'; + class: string; + label: string; + data_attributes: string; + _selectors?: ElementSelectors; + _extractId?: string; +} + +export interface SliderElement { + type: 'slider'; + name: string; + id: string; + value: string; + min: string; + max: string; + step: string; + disabled: 'yes' | 'no'; + class: string; + label: string; + data_attributes: string; + _selectors?: ElementSelectors; + _extractId?: string; +} + export interface CustomElement { type: string; text: string; @@ -154,6 +232,38 @@ export interface CustomElement { _extractId?: string; } +export interface DatatableElement { + type: 'datatable' | 'emap-datatable' | 'jqx-grid' | 'ag-grid'; + text: string; + tag: string; + id: string; + class: string; + // Table structure + rowCount: number; + columnCount: number; + columns: string; + totalRecords: string; + currentPage: string; + totalPages: string; + // Features + hasPagination: boolean; + hasSorting: boolean; + hasFiltering: boolean; + // Data preview + rowSample: string; + // EMAP metadata + emapAction: string; + emapPagePath: string; + emapAppName: string; + emapMetadata: string; + // Standard fields + data_attributes: string; + aria_label: string; + title: string; + _selectors?: ElementSelectors; + _extractId?: string; +} + export interface ExtractedElements { buttons: ButtonElement[]; links: LinkElement[]; @@ -161,6 +271,11 @@ export interface ExtractedElements { inputs: InputElement[]; selects: SelectElement[]; textareas: TextareaElement[]; + checkboxes: CheckboxElement[]; + radios: RadioElement[]; + switches: SwitchElement[]; + sliders: SliderElement[]; + datatables: DatatableElement[]; custom: CustomElement[]; } @@ -172,6 +287,11 @@ export interface ElementSummary { inputs: number; selects: number; textareas: number; + checkboxes: number; + radios: number; + switches: number; + sliders: number; + datatables: number; custom: number; } @@ -201,7 +321,22 @@ export type MessageAction = | 'updateProgress' | 'savePage' | 'highlightElement' - | 'clearHighlight'; + | 'clearHighlight' + | 'highlightAllElements' + | 'clearAllHighlights' + | 'startRecording' + | 'pauseRecording' + | 'resumeRecording' + | 'stopRecording' + | 'getRecordingState' + | 'exportRecording' + | 'clearRecording' + | 'startPlayback' + | 'pausePlayback' + | 'stopPlayback' + | 'seekPlayback' + | 'viewerProgress' + | 'viewerLog'; export interface BaseMessage { action: MessageAction; @@ -240,6 +375,77 @@ export interface ClearHighlightMessage extends BaseMessage { action: 'clearHighlight'; } +export interface HighlightAllElementsMessage extends BaseMessage { + action: 'highlightAllElements'; + elements: Array<{ + selectors: ElementSelectors; + extractId?: string; + }>; + selectorType?: SelectorType; +} + +export interface ClearAllHighlightsMessage extends BaseMessage { + action: 'clearAllHighlights'; +} + +export interface StartRecordingMessage extends BaseMessage { + action: 'startRecording'; +} + +export interface PauseRecordingMessage extends BaseMessage { + action: 'pauseRecording'; +} + +export interface ResumeRecordingMessage extends BaseMessage { + action: 'resumeRecording'; +} + +export interface StopRecordingMessage extends BaseMessage { + action: 'stopRecording'; +} + +export interface GetRecordingStateMessage extends BaseMessage { + action: 'getRecordingState'; +} + +export interface ExportRecordingMessage extends BaseMessage { + action: 'exportRecording'; +} + +export interface ClearRecordingMessage extends BaseMessage { + action: 'clearRecording'; +} + +export interface StartPlaybackMessage extends BaseMessage { + action: 'startPlayback'; + session: RecordingSession; + speed?: number; +} + +export interface PausePlaybackMessage extends BaseMessage { + action: 'pausePlayback'; +} + +export interface StopPlaybackMessage extends BaseMessage { + action: 'stopPlayback'; +} + +export interface SeekPlaybackMessage extends BaseMessage { + action: 'seekPlayback'; + time: number; +} + +export interface ViewerProgressMessage extends BaseMessage { + action: 'viewerProgress'; + data: ViewerProgressUpdate; +} + +export interface ViewerLogMessage extends BaseMessage { + action: 'viewerLog'; + logType: 'start' | 'action' | 'summary' | 'pause' | 'resume' | 'stop'; + data: any; +} + export type ChromeMessage = | ExtractPageMessage | ExtractElementsMessage @@ -247,7 +453,22 @@ export type ChromeMessage = | UpdateProgressMessage | SavePageMessage | HighlightElementMessage - | ClearHighlightMessage; + | ClearHighlightMessage + | HighlightAllElementsMessage + | ClearAllHighlightsMessage + | StartRecordingMessage + | PauseRecordingMessage + | ResumeRecordingMessage + | StopRecordingMessage + | GetRecordingStateMessage + | ExportRecordingMessage + | ClearRecordingMessage + | StartPlaybackMessage + | PausePlaybackMessage + | StopPlaybackMessage + | SeekPlaybackMessage + | ViewerProgressMessage + | ViewerLogMessage; export interface MessageResponse { success: boolean; @@ -268,3 +489,110 @@ export interface StatusState { details?: string; } +// ==================== Viewer Mode Types ==================== + +// User action types +export type UserActionType = + | 'click' | 'dblclick' | 'mousedown' | 'mouseup' + | 'scroll' | 'scrollElement' + | 'input' | 'change' | 'submit' + | 'keydown' | 'keyup' + | 'mouseenter' | 'mouseleave' | 'mousemove' + | 'dragstart' | 'dragend' | 'drop' + | 'focus' | 'blur' + | 'navigation'; // URL change + +// Single user action record +export interface UserAction { + id: string; // Unique identifier + timestamp: number; // Relative timestamp (ms from start) + type: UserActionType; + target?: { // Target element (not for scroll/navigation) + selectors: ElementSelectors; + extractId?: string; + tagName: string; + text: string; // Element text (first 50 chars) + attributes: Record; // Key attributes snapshot + }; + position?: { // Mouse position + x: number; + y: number; + clientX: number; // Viewport coordinates + clientY: number; + }; + scroll?: { // Scroll information + x: number; + y: number; + targetSelector?: string; // Scroll element selector (non-window) + }; + input?: { // Input information + value: string; + isSensitive: boolean; // Is sensitive field + }; + keyboard?: { // Keyboard information + key: string; + code: string; + ctrlKey: boolean; + shiftKey: boolean; + altKey: boolean; + metaKey: boolean; + }; + navigation?: { // Navigation information + from: string; + to: string; + type: 'pushState' | 'popState' | 'hashChange'; + }; + viewport: { // Viewport information + width: number; + height: number; + }; +} + +// Recording session +export interface RecordingSession { + id: string; + startTime: number; // Start timestamp + endTime?: number; // End timestamp + url: string; // Initial URL + title: string; // Page title + duration: number; // Duration (ms) + actions: UserAction[]; + metadata: { + userAgent: string; + screenResolution: string; + timezone: string; + }; +} + +// Recording state +export type RecordingState = 'idle' | 'recording' | 'paused' | 'stopped'; + +// Playback state +export type PlaybackState = 'idle' | 'playing' | 'paused' | 'stopped'; + +// Playback action result +export interface PlaybackActionResult { + actionId: string; + success: boolean; + error?: string; + duration: number; // Execution time (ms) +} + +// Playback progress +export interface PlaybackProgress { + currentIndex: number; + totalActions: number; + currentTime: number; + totalTime: number; + results: PlaybackActionResult[]; +} + +// Viewer progress update +export interface ViewerProgressUpdate { + type: 'recording' | 'playback'; + state: RecordingState | PlaybackState; + actionCount?: number; + duration?: number; + playbackProgress?: PlaybackProgress; +} +