fix: reduce memory usage from unbounded state and eager imports#64
fix: reduce memory usage from unbounded state and eager imports#6427Bslash6 wants to merge 1 commit into
Conversation
- Lazy-import pymupdf4llm and fitz (150-200MB C extensions) only when conversion is needed, not at server startup - Share a single arxiv.Client() via get_arxiv_client() instead of creating a new one per request - Bound conversion_statuses dict by removing entries after completion - Actually clean up PDF files after conversion (was a TODO) - Cap ResearchContext dicts at 50 entries to prevent unbounded growth - Remove dead PaperManager class (never imported by server.py) that pulled in pymupdf4llm and arxiv at module level - Move black from runtime deps to dev (was installing black + deps in production for no reason) All 32 tests pass.
JiwaniZakir
left a comment
There was a problem hiding this comment.
The eviction logic added in prompts/handlers.py is duplicated verbatim at two callsites (explored_papers and paper_analyses), which is worth extracting into a small helper like _evict_oldest(d, max_size) to avoid divergence if _MAX_CONTEXT_ENTRIES semantics ever need to change. Additionally, the cap strategy silently drops the oldest entry with no logging, making it hard to diagnose memory-pressure behavior in production — a single logger.debug line would go a long way.
The get_arxiv_client() singleton in config.py isn't safe under concurrent initialization: two coroutines could both observe _arxiv_client is None before either finishes constructing it, particularly if arxiv.Client() ever suspends or if the server runs tasks in a thread pool. Using a module-level threading.Lock or an asyncio.Lock (initialized at import time, not inside the function) would close this race.
The gutted resources/papers.py is now just a module-level docstring explaining that the class was removed. Keeping an empty file "for potential future use" is unnecessary noise — git history preserves the implementation if it's ever needed again, and the now-empty __init__.py export removal already signals the intent clearly.
Summary
Running this server in k8s, it was OOMing every ~3 hours at a 2Gi memory limit — with zero papers downloaded. Root causes:
arxiv.Client()created fresh on every single request (new HTTP session/connection pool each time, never closed)conversion_statusesdict grows without bound — entries added on every download, never removedResearchContext.explored_papers/paper_analysesdicts grow without boundPaperManagerclass inresources/papers.pyis never imported byserver.pybut pulls inpymupdf4llmandarxivat module levelblack>=25.1.0is in runtimedependenciesinstead ofdevoptional-dependenciesChanges
convert_pdf_to_markdown(), the only function that uses them. Server startup no longer loads ~200MB of C extensions.arxiv.Client()singleton —get_arxiv_client()inconfig.py, reused by download/search/list_papers instead of creating one per request.conversion_statuses— entries removed infinallyblock after conversion completes (success or error).pdf_path.unlink(missing_ok=True)after successful conversion +gc.collect()to nudge C-level allocator.ResearchContext— both dicts capped at 50 entries (FIFO eviction).PaperManager—resources/papers.pygutted (the class was never wired intoserver.py).blackto dev deps — was pulling in black + its dependency tree at runtime for no reason.Test plan
fitzandpymupdf4llmare NOT insys.modulesafter importingserver.pyblack --checkpasses on all files