This repository was archived by the owner on Sep 17, 2025. It is now read-only.
Reorganize dependencies into core and optional groups #152
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Move document processing dependencies to optional 'documents' group - Move indexing dependencies to optional 'indexing' group - Move llama-index dependencies to optional 'llama' group - Keep only core editor functionality as required dependencies - Make MarkdownConverter import optional to avoid import errors - Add helpful error message when document processing is unavailable - Update poetry.lock to reflect new dependency structure This reduces the installation footprint for users who only need basic file editing functionality while maintaining backward compatibility for existing users.
- Made imports optional in md_converter.py with try/except blocks and DOCUMENT_PROCESSING_AVAILABLE flag - Made _CustomMarkdownify class conditional with proper indentation and placeholder when dependencies missing - Added error handling in MarkdownConverter.__init__ to raise helpful error when dependencies unavailable - Fixed compress.py by making libcst imports optional with INDEXING_AVAILABLE flag - Made CompressTransformer class conditional and added placeholder implementations - Fixed get_skeleton function to handle missing dependencies - Added placeholder classes/functions when indexing dependencies unavailable - Added skip condition for integration test that requires indexing dependencies - All 109 unit tests now pass with core-only dependencies Co-authored-by: openhands <[email protected]>
- Move document processing dependencies to default installation (part of core editor) - Move linting dependencies (tree-sitter, grep-ast, flake8) to optional 'linting' group - Move utils dependencies (whatthepatch) to optional 'utils' group - Made linting imports optional in treesitter.py with LINTING_AVAILABLE flag - Made utils imports optional in diff.py with UTILS_AVAILABLE flag - Added conditional class/function definitions with helpful error messages - Updated extras configuration to reflect new organization - Core installation now includes: binaryornot, cachetools, charset-normalizer, pydantic + document processing - Optional groups: linting, utils, indexing, llama, all This makes the core installation lighter while maintaining full functionality through extras. Co-authored-by: openhands <[email protected]>
|
Looks like there are a few issues preventing this PR from being merged!
If you'd like me to help, just leave a comment, like
Feel free to include any additional details that might help me get this PR into a better state. You can manage your notification settings |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR reorganizes the dependencies in
pyproject.tomlto separate core functionality from optional features, significantly reducing the installation footprint for users who only need basic file editing capabilities.Related Issue
Addresses the need to organize dependencies into different groups as discussed - separating core editor functionality from optional document processing and indexing features.
Motivation and Context
The current dependency structure requires all users to install heavy dependencies like pandas, matplotlib, document processing libraries, and LLaMA Index components, even if they only need basic file editing functionality. This creates unnecessary bloat and potential conflicts.
Key Changes:
Core dependencies (9 packages): Only includes packages needed for basic
file_editorfunctionalitybinaryornot,cachetools,charset-normalizer,pydantic(editor core)tree-sitter,tree-sitter-language-pack,grep-ast,flake8(linting)whatthepatch(diff operations)Optional dependency groups:
documents(14 packages): Document processing (PDF, DOCX, PPTX, audio, YouTube transcripts)indexing(6 packages): Code indexing and analysis (pandas, matplotlib, networkx, etc.)llama(3 packages): LLaMA Index integrationall: All optional dependencies combinedBackward compatibility: Made
MarkdownConverterimport optional with helpful error messagesHow Has This Been Tested?
Installation Options:
Does this PR introduce a breaking change?
No breaking changes. This maintains full backward compatibility:
Benefits:
@xingyaoww can click here to continue refining the PR
Fix #153