Fix duplicate_service.py empty-store crash and concurrency bugs - #2035
Conversation
|
@rhoggs-bot-test-account is attempting to deploy a commit to the ritesh Team on Vercel. A member of the Team first needs to authorize it. |
📝 WalkthroughWalkthroughThis PR hardens ChangesThread-safe duplicate detection with atomic persistence
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Hi @Ojas2095! Thanks for the contribution. I have triaged your PR and set it to merge into the
Welcome to the HELPDESK.AI developer family! 🚀💻 |
|
Superb implementation, @Ojas2095! I've successfully resolved all conflicts in your PR and queued it for merging into
Keep up the outstanding work! Let's build together! 🔥 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
backend/services/duplicate_service.py (2)
34-60:⚠️ Potential issue | 🟠 Major | ⚡ Quick winSynchronize
load()initialization to avoid first-call races.Line 34’s guard is unsynchronized, so concurrent first requests can both enter
load(), load the model twice, and append cached tickets more than once. This undermines the intended thread-safety hardening.💡 Suggested fix
class DuplicateService: def __init__(self): self.model = None self._loaded = False self._load_failed = False + self._init_lock = threading.Lock() # In-memory store: list of (ticket_id, embedding, text) self._tickets: list[tuple[str, object, str]] = [] def load(self): """Load the sentence-transformer model and saved tickets.""" - if self._loaded or self._load_failed: - return + if self._loaded or self._load_failed: + return + with self._init_lock: + if self._loaded or self._load_failed: + return - print("[DuplicateService] Loading model...") - try: + print("[DuplicateService] Loading model...") + try: ... - except Exception as e: + except Exception as e: ...
111-158:⚠️ Potential issue | 🔴 Critical | 🏗️ Heavy lift
check_duplicate()return contract now breaks existing request-path consumers.
check_duplicate()now returnsNone(Line 131/138/158) and renamedsimilaritytosimilarity_score(Line 155), but the currentbackend/main.pyflow dereferencesdup_result["is_duplicate"]anddup_result["similarity"]directly. This can crash requests when no duplicate exists.Please either (a) update all callers to handle
None+ new key, or (b) preserve backward compatibility in this method until caller migration is complete.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/services/duplicate_service.py` around lines 111 - 158, The check_duplicate function's changed return contract breaks callers; restore backward compatibility by ensuring check_duplicate always returns a dict (never None) with both the old key "similarity" and the new "similarity_score" plus "is_duplicate" and "duplicate_ticket_id"; when no duplicate is found or the model is unavailable, return {"is_duplicate": False, "duplicate_ticket_id": None, "similarity": 0.0, "similarity_score": 0.0} so existing backend/main.py callers that dereference dup_result["is_duplicate"] and dup_result["similarity"] keep working while also providing the new similarity_score field for migration.
🧹 Nitpick comments (1)
backend/tests/conftest.py (1)
24-24: ⚡ Quick winAvoid global
sys.modulespatch leakage across the whole test suite.Line 24 mutates global import state for all tests. Prefer scoping this through
monkeypatchin a fixture so mocks are reverted automatically per test/session boundary.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/tests/conftest.py` at line 24, The global assignment sys.modules["sentence_transformers"] = _mock_st leaks a mock across the entire test process; replace it with a pytest fixture that uses monkeypatch to scope the replacement (e.g., def mock_sentence_transformers(monkeypatch): monkeypatch.setitem(sys.modules, "sentence_transformers", _mock_st)) and register it either as a normal fixture used by tests or as autouse with the desired scope (function/session) so the mock is reverted automatically; update tests to depend on or rely on that fixture instead of the direct global assignment.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/tests/conftest.py`:
- Around line 12-15: The current special-case branch "if 'VPN' in text" assigns
the same seed value and therefore identical embeddings for all VPN-containing
inputs; change the seed derivation so it's still deterministic but varies per
input (e.g., incorporate the full text or a hash of the text instead of a fixed
42, or combine 42 with sum(ord(c) for c in text) or the ticket id) so different
VPN tickets produce different embeddings; update the code around the "if 'VPN'
in text'" branch and the variable "seed" used by the mock encoder so embeddings
remain stable yet distinct for different texts.
In `@backend/tests/test_duplicate_service.py`:
- Around line 59-60: The two one-line for loops using the variable threads (for
t in threads: t.start() and for t in threads: t.join()) should be expanded to
multi-line for statements to satisfy Ruff E701; replace each single-line form
with a normal block (for t in threads: then newline and an indented t.start())
and do the same for t.join() so the loops are on separate lines and no multiple
statements appear on one line.
---
Outside diff comments:
In `@backend/services/duplicate_service.py`:
- Around line 111-158: The check_duplicate function's changed return contract
breaks callers; restore backward compatibility by ensuring check_duplicate
always returns a dict (never None) with both the old key "similarity" and the
new "similarity_score" plus "is_duplicate" and "duplicate_ticket_id"; when no
duplicate is found or the model is unavailable, return {"is_duplicate": False,
"duplicate_ticket_id": None, "similarity": 0.0, "similarity_score": 0.0} so
existing backend/main.py callers that dereference dup_result["is_duplicate"] and
dup_result["similarity"] keep working while also providing the new
similarity_score field for migration.
---
Nitpick comments:
In `@backend/tests/conftest.py`:
- Line 24: The global assignment sys.modules["sentence_transformers"] = _mock_st
leaks a mock across the entire test process; replace it with a pytest fixture
that uses monkeypatch to scope the replacement (e.g., def
mock_sentence_transformers(monkeypatch): monkeypatch.setitem(sys.modules,
"sentence_transformers", _mock_st)) and register it either as a normal fixture
used by tests or as autouse with the desired scope (function/session) so the
mock is reverted automatically; update tests to depend on or rely on that
fixture instead of the direct global assignment.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 92fc221a-d279-4622-90c8-6cd8f2c5abed
📒 Files selected for processing (5)
backend/data/case_history_cache.jsonbackend/services/duplicate_service.pybackend/tests/__init__.pybackend/tests/conftest.pybackend/tests/test_duplicate_service.py
| if 'VPN' in text: | ||
| seed = 42 | ||
| else: | ||
| seed = sum(ord(c) for c in text) |
There was a problem hiding this comment.
The encoder mock creates tie scores, so “best-match” behavior is not truly validated.
Line 12-15 maps every VPN-containing text to the same seed/embedding. That makes multiple VPN tickets indistinguishable and can hide regressions in ranking logic.
💡 Suggested fix
def _fake_encode(text, convert_to_tensor=False):
- import hashlib, torch
- if 'VPN' in text:
- seed = 42
- else:
- seed = sum(ord(c) for c in text)
+ import hashlib, torch
+ digest = hashlib.sha256(text.encode("utf-8")).hexdigest()
+ seed = int(digest[:8], 16)
torch.manual_seed(seed)
return torch.nn.functional.normalize(torch.randn(1, 384), dim=1).squeeze(0)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if 'VPN' in text: | |
| seed = 42 | |
| else: | |
| seed = sum(ord(c) for c in text) | |
| def _fake_encode(text, convert_to_tensor=False): | |
| import hashlib, torch | |
| digest = hashlib.sha256(text.encode("utf-8")).hexdigest() | |
| seed = int(digest[:8], 16) | |
| torch.manual_seed(seed) | |
| return torch.nn.functional.normalize(torch.randn(1, 384), dim=1).squeeze(0) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/conftest.py` around lines 12 - 15, The current special-case
branch "if 'VPN' in text" assigns the same seed value and therefore identical
embeddings for all VPN-containing inputs; change the seed derivation so it's
still deterministic but varies per input (e.g., incorporate the full text or a
hash of the text instead of a fixed 42, or combine 42 with sum(ord(c) for c in
text) or the ticket id) so different VPN tickets produce different embeddings;
update the code around the "if 'VPN' in text'" branch and the variable "seed"
used by the mock encoder so embeddings remain stable yet distinct for different
texts.
| for t in threads: t.start() | ||
| for t in threads: t.join() |
There was a problem hiding this comment.
Split one-line for statements to satisfy Ruff E701.
Line 59 and Line 60 currently use multiple statements on one line; this violates configured lint rules and can fail CI.
💡 Suggested fix
- for t in threads: t.start()
- for t in threads: t.join()
+ for t in threads:
+ t.start()
+ for t in threads:
+ t.join()📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| for t in threads: t.start() | |
| for t in threads: t.join() | |
| for t in threads: | |
| t.start() | |
| for t in threads: | |
| t.join() |
🧰 Tools
🪛 Ruff (0.15.15)
[error] 59-59: Multiple statements on one line (colon)
(E701)
[error] 60-60: Multiple statements on one line (colon)
(E701)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_duplicate_service.py` around lines 59 - 60, The two
one-line for loops using the variable threads (for t in threads: t.start() and
for t in threads: t.join()) should be expanded to multi-line for statements to
satisfy Ruff E701; replace each single-line form with a normal block (for t in
threads: then newline and an indented t.start()) and do the same for t.join() so
the loops are on separate lines and no multiple statements appear on one line.
Source: Linters/SAST tools
7241c30
into
riteshbonthalakoti:gssoc
|
/claim |
2 participants
Fixes #1824 by introducing empty-store guard, atomic save_to_disk, and threading.Lock in duplicate_service.py. It also introduces tests to prevent regressions.
Summary by CodeRabbit
New Features
Bug Fixes