refactor: restructure microagents system #5886

xingyaoww · 2024-12-28T22:34:41Z

This PR restructures the microagents system to provide a clearer separation between repository-specific instructions and shareable microagents.

Key Changes

Two Sources for Instructions
- Repository Instructions: .openhands/microagents/repo.md in each repository
  - Private to each repository
  - Repository-specific guidelines
  - Team practices and conventions
- Shareable Microagents: OpenHands/microagents/
  - Available to all users
  - Reusable knowledge and workflows
  - Maintained in OpenHands repository

Simplified Hub Structure

OpenHands/microagents/
├── knowledge/  # Keyword-triggered expertise
└── tasks/     # Interactive workflows

Documentation
- Clear separation between private and shareable content
- Better examples and guidelines
- Clearer contribution process
- Loading order explanation
Code Changes
- New microagent models with validation
- Support for both knowledge and task agents
- Improved loading and matching system
- Comprehensive test coverage

Migration Notes

Move repository-specific instructions to .openhands/microagents/repo.md in each repository
Convert shareable agents to new YAML format
Keep private content in repositories, share reusable knowledge in OpenHands

Part of #5538 and #5826

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:3b5523b-nikolaik   --name openhands-app-3b5523b   docker.all-hands.dev/all-hands-ai/openhands:3b5523b

- Move microagents to root level directory - Convert existing agents to YAML format - Add support for repository and keyword triggers - Add template-based workflow support - Remove old micro directory

- Add MicroAgentHub for centralized agent management - Support repository and keyword triggers - Add template-based agent support - Update base.py to use new system

- Add tests for knowledge agent validation - Add tests for template agent validation - Add tests for agent loading and matching - Add tests for template processing

- Keep legacy microagent test - Add environment variable testing to new agents - Add comprehensive tests for new microagent system - Move all tests to tests/unit

- Remove repo/ directory (should be in individual repos) - Move keyword agents to knowledge/ directly - Update documentation to reflect new structure

- Remove official/community split - Rename templates to tasks - Move all agents to top-level categories - Update documentation

- Convert all YAML files to markdown with frontmatter - Improve readability with better formatting - Keep YAML metadata in frontmatter - Add better examples and documentation

- Update format guidelines to show markdown with frontmatter - Add better examples for both agent types - Show recommended structure for content

- Add markdown parsing with YAML frontmatter - Update models to handle markdown content - Rename template to task for clarity - Add better input validation - Update tests for new format

.github/workflows/py-unit-tests.yml

- Use os.walk() to recursively find microagent files in subdirectories - Update CodeActAgent to use correct microagents path - Fix prompt caching tests to work with new structure

microagents/README.md

neubig · 2024-12-31T00:56:59Z

microagents/README.md

+        └── repo.md    # Repository-specific instructions
+```
+
+## 2. Shareable Microagents (Public)


I think that these should be mentioned first, since they are common to all users.

microagents/README.md

tests/unit/test_codeact_agent.py

- Remove config.use_microagents = False from test fixture - Tests now run with microagents enabled, matching real-world usage

- Add brief intro explaining what microagents are - Reorder sections to mention shareable microagents first - Remove vague testing section

microagents/README.md

neubig

LGTM, @xingyaoww could you double-check for final approval and merge if everything looks good?

neubig · 2024-12-31T05:42:42Z

I think the UI related stuff can be handled in a future PR FWIW. And microagents being enabled by default is true in main now right?

Large code changes since last review.

xingyaoww · 2025-01-02T03:45:33Z

I think the UI related stuff can be handled in a future PR FWIW

yep! I'll make sure our existing capability of micro-agent is intact and call it off for this PR.

And microagents being enabled by default is true in main now right?

yep! I'll send off another PR to disable it by default and only enable it for UI

xingyaoww · 2025-01-02T03:46:41Z

@neubig Actually i think this is probably ready for another look (I got failing tests passed locally) and this is working in UI as well

neubig

Overall this looks great to me! Just a few small comments.

microagents/README.md

openhands/microagent/__init__.py

Co-authored-by: Graham Neubig <[email protected]>

neubig

LGTM, thanks for this!

Co-authored-by: openhands <[email protected]> Co-authored-by: Graham Neubig <[email protected]>

openhands-agent added 9 commits December 28, 2024 22:34

refactor: restructure microagents system

f785f88

- Move microagents to root level directory - Convert existing agents to YAML format - Add support for repository and keyword triggers - Add template-based workflow support - Remove old micro directory

feat: implement microagent loading system

31130ab

- Add MicroAgentHub for centralized agent management - Support repository and keyword triggers - Add template-based agent support - Update base.py to use new system

test: add comprehensive tests for microagent system

9388b43

- Add tests for knowledge agent validation - Add tests for template agent validation - Add tests for agent loading and matching - Add tests for template processing

test: merge microagent tests

65f84f1

- Keep legacy microagent test - Add environment variable testing to new agents - Add comprehensive tests for new microagent system - Move all tests to tests/unit

refactor: simplify microagents structure

656e229

- Remove repo/ directory (should be in individual repos) - Move keyword agents to knowledge/ directly - Update documentation to reflect new structure

refactor: simplify microagents structure

8e6005e

- Remove official/community split - Rename templates to tasks - Move all agents to top-level categories - Update documentation

refactor: convert microagents to markdown format

c9a7645

- Convert all YAML files to markdown with frontmatter - Improve readability with better formatting - Keep YAML metadata in frontmatter - Add better examples and documentation

docs: update microagents README with markdown format

f7573cb

- Update format guidelines to show markdown with frontmatter - Add better examples for both agent types - Show recommended structure for content

feat: update microagent system to use markdown

226ae37

- Add markdown parsing with YAML frontmatter - Update models to handle markdown content - Rename template to task for clarity - Add better input validation - Update tests for new format

neubig self-assigned this Dec 30, 2024

openhands-agent added 8 commits December 30, 2024 22:53

fix: update microagents validation for legacy support

d9f85df

style: format microagents.py with ruff

9425a51

style: fix ruff issues

62173b1

style: fix mypy issues

b1844f1

style: fix ruff issues in runtime and tests

2307211

test: fix codeact agent tests

de1e424

fix: update pydantic config for v2

1e253e4

ci: update test workflow to skip docker tests

38418fa

neubig reviewed Dec 31, 2024

View reviewed changes

.github/workflows/py-unit-tests.yml Outdated Show resolved Hide resolved

neubig and others added 2 commits December 31, 2024 09:07

Update .github/workflows/py-unit-tests.yml

94d1a87

fix: update PromptManager to handle new microagents directory structure

4d7709c

- Use os.walk() to recursively find microagent files in subdirectories - Update CodeActAgent to use correct microagents path - Fix prompt caching tests to work with new structure

neubig reviewed Dec 31, 2024

View reviewed changes

openhands-agent and others added 2 commits December 31, 2024 01:02

fix: enable microagents in test_codeact_agent.py

991aee5

- Remove config.use_microagents = False from test fixture - Tests now run with microagents enabled, matching real-world usage

docs: improve microagents README.md

d633578

- Add brief intro explaining what microagents are - Reorder sections to mention shareable microagents first - Remove vague testing section

neubig force-pushed the microagents-restructure branch from f3fa573 to d633578 Compare December 31, 2024 01:06

neubig reviewed Dec 31, 2024

View reviewed changes

microagents/README.md Outdated Show resolved Hide resolved

Update microagents/README.md

d7527db

neubig reviewed Dec 31, 2024

View reviewed changes

neubig marked this pull request as ready for review December 31, 2024 01:08

Merge branch 'main' into microagents-restructure

3d6510f

simplify task input and add additional tasks

362614d

add update test task

66f4430

neubig assigned xingyaoww and unassigned neubig Dec 31, 2024

xingyaoww and others added 7 commits January 1, 2025 22:10

fix fileio

6050d07

move microagent to its own module

3691e7b

add type for microagent

02d9ec0

fix prompt manager

5b91c0a

fix microagent utils as well

4f2dbe7

fix import

bbc538c

Merge branch 'main' into microagents-restructure

37ead7b

xingyaoww requested a review from rbren January 2, 2025 03:46

neubig self-requested a review January 2, 2025 05:01

neubig assigned neubig and unassigned xingyaoww Jan 2, 2025

neubig reviewed Jan 2, 2025

View reviewed changes

microagents/README.md Outdated Show resolved Hide resolved

microagents/README.md Outdated Show resolved Hide resolved

microagents/README.md Outdated Show resolved Hide resolved

openhands/microagent/__init__.py Outdated Show resolved Hide resolved

xingyaoww and others added 6 commits January 3, 2025 00:00

Update microagents/README.md

6e1dadd

Co-authored-by: Graham Neubig <[email protected]>

Update microagents/README.md

e484bca

Co-authored-by: Graham Neubig <[email protected]>

Update microagents/README.md

e923c47

Co-authored-by: Graham Neubig <[email protected]>

Merge branch 'main' into microagents-restructure

7899a22

break microagent init to multiple files

e99348a

Merge branch 'main' into microagents-restructure

3b5523b

neubig approved these changes Jan 2, 2025

View reviewed changes

neubig merged commit c1b514e into main Jan 2, 2025
14 checks passed

neubig added a commit that referenced this pull request Jan 2, 2025

refactor: restructure microagents system (#5886)

c1b514e

Co-authored-by: openhands <[email protected]> Co-authored-by: Graham Neubig <[email protected]>

neubig deleted the microagents-restructure branch January 2, 2025 22:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: restructure microagents system #5886

refactor: restructure microagents system #5886

xingyaoww commented Dec 28, 2024 •

edited by github-actions bot

Loading

neubig Dec 31, 2024

neubig left a comment •

edited

Loading

neubig commented Dec 31, 2024 •

edited

Loading

xingyaoww commented Jan 2, 2025

xingyaoww commented Jan 2, 2025 •

edited

Loading

neubig left a comment

neubig left a comment

refactor: restructure microagents system #5886

refactor: restructure microagents system #5886

Conversation

xingyaoww commented Dec 28, 2024 • edited by github-actions bot Loading

Key Changes

Migration Notes

neubig Dec 31, 2024

Choose a reason for hiding this comment

neubig left a comment • edited Loading

Choose a reason for hiding this comment

neubig commented Dec 31, 2024 • edited Loading

xingyaoww commented Jan 2, 2025

xingyaoww commented Jan 2, 2025 • edited Loading

neubig left a comment

Choose a reason for hiding this comment

neubig left a comment

Choose a reason for hiding this comment

xingyaoww commented Dec 28, 2024 •

edited by github-actions bot

Loading

neubig left a comment •

edited

Loading

neubig commented Dec 31, 2024 •

edited

Loading

xingyaoww commented Jan 2, 2025 •

edited

Loading