Skip to content

[Feature]: Process-level write lock for build/update/postprocess/embed #1073

Open

Description

@nhuphuoc-bic

Problem

Nothing stops two processes from writing the same .code-review-graph/graph.db at once. For example, a build started by a git hook can run while a CLI update or the MCP build_or_update_graph_tool is also running. SQLite serializes each transaction, but a build or update spans many commits, so one writer can post-process (flows, communities, FTS) a graph that the other writer has only partly written. Today the only guard is busy_timeout=5000, so a second writer stalls for up to 5s at each lock wait when it should fail fast. (v2.3.9)

Proposed solution

Before writing, every command that writes the graph takes a non-blocking exclusive lock on a fixed file. That covers the CLI build, update, postprocess, embed, forget and watch, and the matching MCP tools:

fd = os.open(data_dir / "write.lock", os.O_CREAT | os.O_RDWR)
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)  # BlockingIOError -> "another write is running", exit non-zero

The kernel releases a flock when its holder dies, so stale locks never need cleaning up.

The path should be documented. External tooling (Claude Code hooks, git hooks) can then take the same lock with flock -n .code-review-graph/write.lock code-review-graph build, and both sides exclude each other.

Affected area

Graph store / SQLite

Alternatives considered

We already take this lock in our wrapper scripts. It cannot cover a CLI command typed by hand, or a write from an MCP server that is already running.

Activity

  1. Ariosolzq commented on Oct 8, 2026

    @Ariosolzq
    Contributor

    We reproduced the concurrent-write problem described here on code-review-graph 2.3.9 in a large repository.

    What happened

    During a cold graph bootstrap, multiple MCP clients started concurrently and each attempted to build/update the same .code-review-graph/graph.db.

    The resulting database had:

    • 125,355 live nodes
    • 949,204 live edges
    • 48,443 live nodes missing from nodes_fts
    • 48,443 ghost FTS entries pointing to deleted node rowids

    The graph data itself was valid, but the FTS index was inconsistent, causing valid symbols to return empty or incorrect search results.

    A fresh isolated build with the same CRG version and repository produced the same node/edge counts with:

    • 0 missing FTS entries
    • 0 ghost FTS entries

    Running postprocess also repaired the affected index.

    Why this matters

    SQLite transaction locking prevents some direct write conflicts, but it does not provide process-level serialization across the full build/update/post-processing lifecycle.

    Multiple writers can therefore interleave graph mutations and derived-index processing.

    A process-level single-writer lock as proposed in this issue would prevent this class of failure.

  2. Bakul2006 commented on Oct 17, 2026

    @Bakul2006
    Contributor

    Hey @tirth8205 the issue sounds to be interesting....
    Before I start implementing this, I wanted to check whether the proposed flock approach is the right direction.

    The reproduction suggests that concurrent writers can leave the graph's FTS index inconsistent, even when the underlying node and edge counts remain valid. Since SQLite transactions don't serialize the entire build/update/post-processing lifecycle, a shared process-level lock seems like a reasonable solution.

    My main questions are:

    1. Would you prefer a shared, non-blocking flock on .code-review-graph/write.lock, or is there an existing locking abstraction or architectural approach you'd prefer us to use?
    2. Should lock contention always fail immediately with a non-zero exit status, including for MCP requests, or should any operations wait or retry?
    3. Are there any platform compatibility requirements or existing conventions I should account for before implementing this?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions