fix(serve): prefer exact label match in MCP shortest_path / query_graph#879
fix(serve): prefer exact label match in MCP shortest_path / query_graph#879josephmcknight-bot wants to merge 2 commits into
Conversation
…esolved-label echo
Three new behaviors covered (all currently failing):
- _find_node('audit') must rank the literal 'audit' node ahead of
substring matches like 'audit_log' / 'audit()'. Falls back to
substring only when no exact hit exists.
- _score_nodes must apply an exact-match bonus so the literal label
strictly outranks substring matches, including function-suffix
labels like 'audit()'.
- _tool_shortest_path must echo resolved labels (Resolving 'X' ->
node 'Y') so silent substring coercion is visible to callers.
Surfaced via a new _build_shortest_path_handler factory so the
closure is testable without spinning up an MCP stdio server.
Today an MCP call like:
shortest_path('Repo content audit - 2026-05-15', X)
silently coerces the source to the 'audit()' function node (a
high-degree substring hit) instead of the literal audit document the
caller meant. No warning. Operator-impacting - cost ~1 hour of
misdiagnosis during a recent audit before get_node was used to pin
the actual node ID.
Three changes in graphify/serve.py:
1. _find_node now returns exact label matches first, falling back to
substring only when no exact hit exists. Mirrors how _tool_get_node
already orders matches. Strips trailing '()' so a query for 'audit'
still matches the function-style label 'audit()'.
2. _score_nodes applies a +10 bonus for exact label match. The bonus
dwarfs the substring (+1) and source-file (+0.5) weights, so the
literal 'audit' node strictly outranks 'audit_log', 'audit_trail',
and 'audit_runner' substring matches.
3. _tool_shortest_path echoes resolved labels:
Resolving 'audit' -> node 'audit'
Resolving 'cluster' -> node 'cluster'
Shortest path (...)
so any remaining substring coercion is visible to the caller
instead of disappearing into the path summary.
Refactor: the shortest_path closure body is extracted to
_build_shortest_path_handler(G) so it can be unit-tested without
spinning up an MCP stdio server. serve() invokes the factory at the
same place the inline definition used to live; behaviour is identical.
Tests: 6 new cases in tests/test_serve.py (23 serve tests total,
all green). Full suite: 258 passed / 41 pre-existing failures (all
missing-optional-deps; identical to baseline). No MCP tool schemas
change - existing clients work unchanged.
Out of scope: get_node already orders exact-then-substring correctly
and is unchanged. No new MCP tools added.
safishamsi
left a comment
There was a problem hiding this comment.
Thanks for the contribution! Unfortunately this PR was built against a stale baseline and would regress several improvements that landed on main since the branch was cut.
Specifically, main already has:
- 3-tier priority scoring (
1000/100/1weights for exact/prefix/substring matches) — the scoring in this PR would replace that with a simpler scheme that produces worse ranking - Diacritic-insensitive normalization — this PR's normalization step doesn't include it, so accented characters would fall back to lower quality matches
- Ambiguity warnings — main surfaces uncertain matches to the user; this PR silently drops them
Rather than closing, I'd suggest rebasing onto current v8 (or main) and only keeping the parts of this PR that add new behavior not already present. Happy to point out exactly which hunks to keep if that would help — just let us know.
|
Thanks for this fix - the exact-match bonus and test structure are excellent. However this PR is based on |
Why
Today an MCP call like:
silently coerces the source to the
audit()function node (a high-degree substring hit) instead of the literal audit document the caller meant. No warning is emitted; the path summary just shows an unrelated walk through code. Operator-impacting - cost ~1 hour of misdiagnosis during a real audit beforeget_nodewas used to pin the actual node ID. Any downstream agent usingshortest_pathto verify architectural coupling is exposed to the same silent-coercion failure.What changed
Three changes in
graphify/serve.py(file-line references againstmainHEAD; in publishedgraphifyy==0.7.15the lines are 235 / 54 / 451 respectively):_find_nodeexact-then-substring (serve.py:96here). Returns exact label matches first, falls back to substring only when no exact hit exists. Mirrors how_tool_get_nodealready orders its matches. Strips trailing()so a query forauditstill matches the function-style labelaudit()._score_nodesexact-match bonus (serve.py:34here). Applies a +10 bonus for exact label match. The bonus dwarfs the substring (+1) and source-file (+0.5) weights, so the literalauditnode strictly outranksaudit_log,audit_trail, andaudit_runnersubstring matches._tool_shortest_pathechoes resolved labels (serve.py:266here). Output now starts with:so any remaining substring coercion is visible to the caller instead of disappearing into the path summary.
Refactor
The
shortest_pathclosure body is extracted into a new_build_shortest_path_handler(G)factory so the closure can be unit-tested without spinning up an MCP stdio server.serve()invokes the factory at the same point the inline definition used to live; runtime behaviour is identical.Tests
6 new cases added to
tests/test_serve.py(23 serve tests total, all green):test_find_node_prefers_exact_match_over_substring- literalauditranks ahead ofaudit_log/audit().test_find_node_falls_back_to_substring_when_no_exact_match- regression guard for the fallback path.test_score_nodes_exact_match_outranks_substring- asserts strict (not tied) lead.test_score_nodes_exact_match_handles_function_suffix-auditmatchesaudit().test_resolve_label_returns_label_for_resolved_node- helper, including fallback to ID.test_tool_shortest_path_echoes_resolved_labels- asserts the newResolving ... -> node ...prefix.Full suite: 258 passed / 41 pre-existing failures (all
ModuleNotFoundErroron optional[mcp,pdf,watch]extras when the environment lacksgraspologic/pypdf/etc; identical to themainbaseline before this PR).Out of scope / non-goals
_tool_get_nodealready orders exact-then-substring correctly. Not changed.shortest_pathis additive (two prefix lines), and the resolved labels are now correct rather than coerced.Notes
f04c912); implementation in commite7cbfb9.