Claude/extract special items 016qb pai5y dfh qw h7r uck cc k#47
Open
haowenluo wants to merge 4 commits into
Open
Claude/extract special items 016qb pai5y dfh qw h7r uck cc k#47haowenluo wants to merge 4 commits into
haowenluo wants to merge 4 commits into
Conversation
The inventory display was confusing because it showed: - "Downloaded Filings: 10-K: 5,618" (unique filings from metadata) - "RAW Filings: 10-K: 85,019 files" (all files on disk) This caused confusion about why there were ~15 files per filing. Changes: - Added _count_unique_filings() method to count unique filings by accession number - Updated RAW Filings section to show both unique filings and total files - Added clear labels: "(from metadata)" and "(on disk)" - Added comparison section showing metadata vs disk discrepancies - Added explanatory notes about what each metric means - Shows files-per-filing ratio to make the relationship clear Now displays: "10-K: 5,618 unique filings (85,019 total files, ~15.1 files/filing)" This makes it immediately clear that you have 5,618 unique filings that are stored as 85,019 total files on disk.
…DnmaNQtFpWqjL8B6Bwq Fix inventory display to clarify file counts vs unique filings
This commit implements automated extraction of special/nonrecurring items from financial statements, including restructuring charges, impairments, litigation settlements, and discontinued operations. Key changes: - Added extract_special_items() method with keyword-based pattern matching - Implemented monetary amount extraction and footnote reference linking - Added confidence scoring system (0.0-1.0) based on context - Inserted extraction before table removal to preserve disclosure context - Added special_items array to JSON output with structured fields - Added comprehensive configuration in config.json - Updated README with documentation and usage examples - Added unit tests for special items extraction Features: - Scans Item 8 (Financial Statements) with optional Item 7 (MD&A) scanning - Extracts amounts in various formats ($X million, $X.X billion, etc.) - Captures footnote references (Note X, (X), etc.) - Provides context snippets for each finding - Includes anchor hash for source traceability - Supports customizable keywords and confidence thresholds - Debug logging mode for development and tuning Implementation follows user-provided design principles: - Pre-cleaning extraction to preserve table and footnote context - Lightweight anchor system for post-cleaning source mapping - Backward compatibility with special_items_enabled flag - Both raw text and normalized numeric fields for amounts
Enhances special items extraction with two new categories to provide comprehensive coverage of nonrecurring items commonly found in 10-K financial statements. New categories: - acquisition: Captures M&A-related expenses including transaction costs, integration costs, advisory fees, merger expenses, earnout adjustments, and purchase price adjustments - asset_sale: Captures gains/losses from asset sales and divestitures including asset disposal, sale of assets, and disposal proceeds Changes: - Added 12 M&A-related keywords to "acquisition" category - Added 8 asset sale keywords to "asset_sale" category - Reorganized "other" category to focus on remaining items - Updated confidence scoring: acquisition=0.3 (medium), asset_sale=0.25 - Enhanced README documentation with new category descriptions - Updated unit tests to validate new categories - All tests passing successfully Test results show proper detection: - Acquisition keywords: transaction costs, advisory fees, integration - Asset sale keywords: gain on sale, divestiture, asset disposal - Appropriate confidence scores (0.55-0.80 range)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.