Skip to content

feat(storage): carry the git data ref of a git-backed Dolt remote - #6521

Merged
julianknutsen merged 3 commits into
gastownhall:mainfrom
sdghbtn:feat/dolt-remote-git-ref-storage
Sep 27, 2026
Merged

julianknutsen merged 3 commits into
gastownhall:mainfrom
sdghbtn:feat/dolt-remote-git-ref-storage

Conversation

@sdghbtn

@sdghbtn sdghbtn commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Storage primitives learn which git ref a git-backed Dolt remote keeps its data on, so the application layer can offer --ref in a follow-up PR without reaching across layers.

  • storage.RemoteInfo gains Ref, JSON ref, omitted when empty. Empty means Dolt's default refs/dolt/data; storage.RemoteRefsMatch and storage.EffectiveGitDataRef compare and resolve refs with that meaning, GitRefParam / DefaultGitDataRef name dolt's parameter and default, and storage.GitRefFromParamsJSON reads the parameter out of a dolt_remotes params value. The proxied-server remote surface (domain.Remote, RemoteSQLRepository.AddRemoteWithRef, DoltRemoteUseCase.CreateRemoteWithRef) carries the ref the same way, so bd init in that mode can use it from cmd/bd. Federation peers (AddFederationPeer, FederationPeer) stay on the default ref; nothing in this pair of PRs changes them.
  • RemoteStore.AddRemoteWithRef on both stores passes a non-empty ref to DOLT_REMOTE('add', '--ref', ...). AddRemote is unchanged and is AddRemoteWithRef with an empty ref.
  • versioncontrolops.DoltCloneWithRef passes the ref to DOLT_CLONE('--ref', ...); dolt.BootstrapFromRemoteWithDBRef adds --ref to the dolt clone argv on the CLI clone path. The existing functions delegate with an empty ref.
  • versioncontrolops.ListRemotes reads the ref back from the params column of dolt_remotes, key git_ref; doltutil.PersistedRemotes reads it from repo_state.json. A remote without the key lists an empty ref. A params value that is not JSON is an error naming the remote, never a silent default.
  • The CLI mirror that server-mode push, pull, and fetch shell out through is compared on URL and ref (hasMatchingCLIRemote, ensureMatchingCLIRemote, doltutil.EnsureCLIRemote) and re-materialized with dolt remote add --ref when the ref differs, so a mirror on the default ref cannot push a custom-ref database to refs/dolt/data. Three guards for the layouts bd manages, where the dolt CLI in that directory is proxied to the running sql-server: when the proxied listing shows nothing and a ref is expected, the check reads the server's own repo_state.json in that directory and treats a matching URL and ref as in place; in the cold-start fallback, where SQL lists nothing and the state file has the remote, a remote with a ref is re-added over bd's SQL connection with --ref, which the procedure accepts where the proxied CLI does not, so the server the transfer delegates to knows the remote; "already exists" is accepted, and the re-add runs only when this bd owns the served directory, localActiveDatabaseDir, the same proof resolveLocalActiveDatabaseDir demands elsewhere. For any other server a client-local state file is not written into the server: the route stops and names the DOLT_REMOTE("add", "--ref", ...) call to run against that server through bd sql, as one shell-quoted word with the values as escaped SQL literals. The stacked PR's bd dolt remote add --ref cannot be that remedy: it finds the persisted remote, sees URL and ref match, and returns without a SQL add. And before a ref-carrying re-materialization removes an existing mirror remote, a throwaway remote is added and removed with the same URL and ref, so dolt's refusal of remote parameters over the proxy is returned before anything is deleted. Probe remotes left by an interrupted run are swept before every change to the mirror, whatever ref it moves to: a remote named exactly as the probe names them, bd-ref-probe-<pid>, whose pid is no longer running, or this process's own name; a remote that only shares the prefix, and the probe of a run still alive, are left alone; the liveness check is kill(pid, 0) on unix, only ESRCH meaning dead, and OpenProcess plus a zero-timeout wait on Windows, cross-compiled here and not run; on any other platform only the own name is swept. Removing another run's leftover is best-effort: a removal that fails is tried again at the next change, and a pid an unrelated process has since taken keeps the leftover until that pid exits; the own-name leftover must go, and its removal failing is an error. A state file that cannot be read at sweep time is an error naming the read, not a report of the server refusing parameters, and a ref that fails the argument check below is refused before the mirror is touched. A remote on the default ref keeps today's path in all three places. The route decider and the cold-start re-add are proven at the store seam with sqlmock and a state file; the live transfer through a served directory is not exercised here, there is no dolt sql-server in this environment, and is the one claim in this PR resting on the code walk rather than a run.

Choices where more than one shape was possible:

  • A new method rather than a wider AddRemote: only DoltStore and EmbeddedDoltStore implement RemoteStore, no fake does, and the conformance suite calls no RemoteStore method, so the addition is free of churn there; keeping AddRemote as it is keeps every cmd/bd caller untouched in this PR.
  • The ref is passed through verbatim: no expansion of a bare name and no ref-name rules at this layer. An empty or whitespace ref means the default and never reaches dolt. A ref on a non-git remote is dolt's own refusal, returned unchanged, tested on the real engine and, with the CLI present, on the real CLI. Before a ref becomes a dolt command-line argument it is checked for a space, an ASCII control character, DEL, and a leading dash (doltutil.ValidateGitDataRefArg): the first three are bytes git's ref rules refuse, the dash is so the value can never be mistaken for an option; dolt accepts all four through exec, so this is bd policy at the argv boundary, not a ref-name check. A non-ASCII space such as U+00A0 is a valid ref byte for git and for dolt and passes, since this layer adds no ref-name rules.
  • SELECT name, url, params FROM dolt_remotes with the JSON parsed in Go rather than JSON_EXTRACT in SQL, so the query has no dependency on JSON function support in the server the client is talking to. The real-engine test proves the parse on the embedded engine for a remote with no parameters and for git_ref in each shape; the embedded DOLT_REMOTE procedure accepts no other parameter, so an unrelated parameter cannot be created there. A params object with an unrelated key, the NULL and "null" renderings, and a malformed value are covered by sqlmock, and the on-disk shape by a repo_state.json fixture.
  • The mirror's ref is read from the mirror directory's repo_state.json, not from dolt remote -v: the listing's rendering of remote parameters is not a documented format, and at GH#2118 cold start the proxied listing is empty, which is the case where the state file is the only source.

Why

Part of #6261. Dolt supports a git remote whose data sits on a ref other than refs/dolt/data (git_ref remote parameter, --ref on dolt remote add, dolt clone, DOLT_REMOTE, and DOLT_CLONE); bd never passes it through, so a host that only accepts pushes under refs/heads/ cannot be used, and neither can one repository holding several Dolt databases on refs such as refs/dolt/units/<key>. This PR is the storage half; the commands, the sync.remote-ref config key, and the docs follow in a stacked PR.

Nothing changes for a remote without a ref: the SQL for add, clone, push, and pull is the same text as before, the mirror comparison degrades to today's URL comparison, and ListRemotes returns the same rows with an extra empty field. One exception, deliberate: the CLI-mirror check reads the mirror directory's repo_state.json unconditionally and fails closed when the file cannot be parsed, where main went on to the URL comparison. An unreadable file could be hiding a custom ref on the mirror, and skipping the read when the expected ref is empty would reopen the wrong-ref push this PR closes.

Verification

The fixtures that exercise ref comparison or parsing carry both shapes, refs/heads/issue-data and refs/dolt/units/team-12542, so both pass through unchanged; tests where the ref is only a bind parameter or an argv element use one.

./scripts/test.sh ./internal/storage/ ./internal/storage/versioncontrolops/... ./internal/storage/doltutil/... ./internal/storage/domain/...   # ok
./scripts/test.sh -run 'TestDoltCloneArgs|TestBootstrapCloneCmdEnvIsGuarded|TestEnsureMatchingCLIRemote|TestSQLCapableCLIRouting|TestPrepareCLIRouteForGitProtocol|TestHasMatchingCLIRemote|TestBootstrapFromRemote' ./internal/storage/dolt/   # ok
./scripts/test.sh -run TestRemoteGitDataRefOnEngine ./internal/storage/embeddeddolt/   # ok, 11.6s: real engine, both shapes: add and list from SQL and repo_state.json, --ref refused on file://, push lands on the ref and not on refs/dolt/data, DOLT_CLONE --ref records the ref on origin
BEADS_TEST_ENV_RUN_DOLT=1 ./scripts/test.sh -run 'TestCLIMirror|TestPersistedRemotes|TestValidateGitDataRefArg|TestLeftoverProbe' ./internal/storage/doltutil/   # ok with dolt 2.3.2 on PATH (skips the CLI cases without): the mirror follows the ref, a refused probe leaves the remote in place, the sweep runs before every mirror change and removes this process's leftover and a dead run's while leaving a live run's probe and a user's bd-ref-probe-backup, an unreadable state file and a failed own-leftover removal are reported as such, an invalid ref is refused before the mirror is touched, over an existing remote too; the dead-pid assertions follow the platform's liveness check, so the suite holds where only the own name is swept
./scripts/test.sh -run 'TestColdStartReaddRemedyQuotes|TestPrepareCLIRouteForGitProtocol' ./internal/storage/dolt/   # ok: the remedy is one shell word with an apostrophe, $HOME, and a backtick in the URL inert
BEADS_TEST_ENV_RUN_DOLT=1 ./scripts/test.sh -run 'TestBootstrapFromRemote|TestDoltStore_hasPersistedCLIRemote|TestEnsureMatchingCLIRemote|TestSQLCapableCLIRouting' ./internal/storage/dolt/   # ok with dolt on PATH
go vet ./...   # ok (gms_pure_go)
golangci-lint run --config=.golangci.yml --build-tags=gms_pure_go ./internal/storage/...   # 0 issues
make test   # ok apart from one failure in scripts/repro-dolt-prod-timeouts, untouched by this PR and green when run alone

@sdghbtn
sdghbtn force-pushed the feat/dolt-remote-git-ref-storage branch 5 times, most recently from aad43de to 0c0a57d Compare September 13, 2026 10:09
@sdghbtn

sdghbtn commented Sep 13, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review before this leaves draft: three rounds under each of two models (six in all), each round with a fresh context in a throwaway worktree of the branch, the diff, the body, and the prior rounds' dispositions.

Findings: 33 in total (4 blocker, 7 major, the rest minor or question). 31 accepted and fixed on the branch, 1 acknowledged (CI has not run on the PR until a maintainer approves the workflow), 1 open with the author. The main changes the rounds drove: the missing //go:build cgo on the engine test; the CLI-mirror check falling back to the served directory's own state file, the cold-start re-add of a ref remote over SQL gated on proven ownership of the served directory, and a probe remote so a proxied dolt remote add --ref refusal is returned before anything is deleted; FindCLIRemoteRef and PersistedRemotes refusing to read an odd state file as the default ref; the proxied-server remote surface carrying the ref; a route-decider test at the store seam, pin tests against dolt's constants, and the body corrected wherever it overstated what the tests prove (one claim stays a code walk: the live transfer through a served directory, which this environment cannot run).

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 0c0a57d45 on main (merge-base f56632adcf). Sorry for the wait on this one.

Ran, all ok: go build ./..., go vet ./internal/storage/..., go test ./internal/storage/ ./internal/storage/versioncontrolops/... ./internal/storage/doltutil/... ./internal/storage/domain/..., -run TestRemoteGitDataRefOnEngine ./internal/storage/embeddeddolt/ (real engine), and with BEADS_TEST_ENV_RUN_DOLT=1 on dolt 2.3.1: TestCLIMirrorFollowsGitDataRef, TestCLIMirrorRefRefusedForNonGitURL, TestPersistedRemotes.

The shape is right and the layering argument in the body holds up: a new method instead of a wider AddRemote, the ref bound as a ? everywhere it goes over SQL, the JSON parsed in Go. Two small things need to change before this stands on its own; the rest are notes.

1. MINOR, please fix — the cold-start guard tells the operator to run a command that does not exist at this commit

internal/storage/dolt/store.go:3671:

"... this bd does not own the server's data directory; re-add it with: bd dolt remote add %s %s --ref %s"

bd dolt remote add is cobra.ExactArgs(2) with one flag, --allow-git-origin (cmd/bd/dolt.go:1603-1608); --ref arrives only with #6525, which is a draft. This PR is meant to merge first, and the guard is reachable without #6525 — a ref remote made with dolt remote add --ref directly, on a server this bd does not own, at GH#2118 cold start. That operator is handed unknown flag: --ref as the way out. cli_mirror_ref_test.go:186 asserts the string, so the test pins it.

Either name what works today (the CALL DOLT_REMOTE('add', '--ref', …) against that server, or dolt remote add --ref in the server's directory), or hold the bd spelling for #6525 and change the message there together with its test.

2. MINOR, please fix — "probes left by an interrupted run are swept" is not what the code does

doltutil/remotes.go:264-271 removes a leftover only when it is named bd-ref-probe-<this pid>, and the comment above it says so. A run killed between the add and the remove leaves bd-ref-probe-<old pid> in repo_state.json, and nothing later matches it short of pid reuse. cliRemoteLock is a sync.Mutex, in-process only, so I can see why you did not sweep by prefix — another bd's live probe could be the thing swept (that process would then fail on its own RemoveCLIRemote(probe) before touching the real remote: spurious, but safe). Your call between:

  • sweep the prefix and accept that spurious failure, or
  • keep own-pid only and correct the body, and say in the comment that a dead run's probe stays until removed by hand.

One related wrinkle: when PersistedRemotes errors, the sweep is skipped silently, and a same-pid leftover then fails the add under the "a dolt sql-server serving this directory refuses remote parameters" wording, which would be the wrong diagnosis.

Notes, not asks

  • "Nothing changes for a remote without a ref" has one exception, and I think it is the right one. EnsureCLIRemote (remotes.go:316-321) and hasMatchingCLIRemote (store.go:3534) now read repo_state.json unconditionally, so a no-ref user with a truncated state file gets an error where main went on to the URL check. Failing closed is correct — an unreadable file could be hiding a custom ref on the mirror, and skipping the read when the expected ref is empty would reopen exactly the wrong-ref push this PR closes. Worth a clause in the body so the claim is exact.
  • PersistedRemotes now fails whole on one malformed params (Params map[string]any; a non-object value fails the unmarshal where the field used to be ignored). dolt always writes an object ("params": {} on a plain remote — checked on 2.3.1), so this takes a hand-edited file to hit; the cost if hit is that federation.go:249-254 warns "assuming none". json.RawMessage plus a per-remote error would keep the blast radius to the one remote. Low priority.
  • Two sources for one remote. current comes from dolt remote -v, currentRef from the state file. On 2.3.1 the listing already carries the params:
    gitr git+https://example.com/repo.git {"git_ref": "refs/heads/issue-data"}
    
    and ListCLIRemotes drops everything after parts[1]. Reading the ref from the same listing (mind the space inside the JSON when splitting) would make URL and ref one observation, and it is the proxied server's own answer rather than the file's. I could not construct a divergence that guard 1 does not already cover, so this is a simplification to consider, not a defect.
  • bd dolt remote reset-data is ref-blind (cmd/bd/dolt_remote_reset_data.go:26,249: pins refs/dolt/data, takes r.URL, never looks at r.Ref). With several databases in one repository — this PR's own motivating layout — resetting a remote on refs/dolt/units/<key> would delete the default-ref database's data plane and leave the target's in place. Out of scope here by your own cmd/bd line, and I see #6525 touches that file; I have not read it yet and will check for this there. Flagging it so it is on the record for the stack.
  • ValidateGitDataRefArg lets U+00A0 through. Not exploitable (no leading dash, git refuses it); mentioning only because the doc comment says "whitespace".

Checked and clean

  • Every path where the ref becomes a dolt argv element validates it: AddCLIRemoteWithRef (which the probe goes through) and doltCloneArgs via BootstrapFromRemoteWithDBRef. Everything else binds it as a parameter.
  • No mirror call site drops the ref (credentials.go, federation.go:639, store.go:3646,3678).
  • The ownership proof holds: localActiveDatabaseDir is empty for gateway, proxied, socket, TLS, non-localhost and env-port configurations, so a client of an external server cannot reach the cold-start SQL re-add.
  • params is in dolt_remotes in the pinned engine (dtables/remotes_table.go:81) and in 2.3.1's repo_state.json; the wider SELECT is safe.
  • The probe-before-remove ordering is real and TestCLIMirrorRefusedProbeLeavesRemoteInPlace would catch a reorder.

Requesting changes for 1 and 2 only; both are a few lines. I'll re-read on the push.

@julianknutsen julianknutsen added the status/needs-triage Awaiting initial PM triage label Sep 19, 2026
@sdghbtn
sdghbtn force-pushed the feat/dolt-remote-git-ref-storage branch from 0c0a57d to efe18a8 Compare September 23, 2026 02:34
@sdghbtn

sdghbtn commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

@bee-ghosttrack Thanks for the careful read, and for running the CLI tests on a real dolt. Pushed as efe18a802, rebased onto main at 7ccb961f7; the rebase was clean, main touched store.go and embeddeddolt/version_control.go in other hunks. Since 0c0a57d45 the change sits in the two places you named plus the whitespace check: store.go and its test for the remedy, doltutil/remotes.go with two small probe_pid_*.go files for the sweep, and the doltutil tests. git range-diff f56632adc..0c0a57d45 7ccb961f7..efe18a802 shows it.

1. The cold-start remedy. Taken: the message now names what works at this commit, the procedure call over bd's connection to that server: re-add it on that server with: bd sql 'CALL DOLT_REMOTE("add", "--ref", "<ref>", "<name>", "<url>")'. It is built as one shell-quoted word with the values as escaped SQL literals, through doltutil.ShellQuote, the same fix 72fc86cf1 made for an earlier pasted command, so an apostrophe or a $ in a URL cannot change the command; TestColdStartReaddRemedyQuotes pins that. I did not take dolt remote add --ref in the server's directory, because that is the proxied CLI your point 2 and the probe are about: it refuses remote parameters while the server runs. I also ended up not holding the bd dolt remote add --ref spelling for #6525: ensureDoltRemoteGuarded there falls back to the persisted remotes in exactly this GH#2118 window, finds the same URL and ref, and returns success without a SQL add, so that command would not repair the condition the message describes. bd sql is available in server mode, the only mode that opens the store this guard lives in, and it either registers the remote or gets "already exists" back, which means the window has closed.

2. The probe sweep. A sweep, narrower than a prefix, and it now runs before every change to the mirror, not only ahead of a probe. sweepLeftoverProbes removes a remote named exactly as the probe names its own, bd-ref-probe-<pid> in canonical decimal and nothing else, when that pid is no longer running, plus a leftover under this process's own name. So a run killed between its add and its remove is cleaned up at the next re-materialization of that mirror, whatever ref it moves to. For another run's leftover that is best-effort: a removal that fails is tried again at the next change, and a pid an unrelated process has since taken keeps the leftover until that pid exits. A live run's probe is left alone, so the concurrent case you described gets no spurious failure; and a remote a user happened to name bd-ref-probe-backup is not touched. Liveness is kill(pid, 0) as internal/doltserver does, read more strictly: only ESRCH means dead, EPERM is alive; on Windows it is the OpenProcess plus zero-timeout wait internal/doltserver uses there, cross-compiled here and not run; on any other platform only the own name is swept. The body says this. The PersistedRemotes error is no longer swallowed: the sweep returns read the recorded remotes in <dir> before changing the CLI mirror: ..., and a leftover under this process's own name whose removal fails is returned as remove leftover ref probe remote ..., so the add can no longer be misread as the server's refusal. In practice that read error is hard to reach, since EnsureCLIRemote read the same file through FindCLIRemoteRef a moment earlier and refused on failure; the unreadable-state test calls sweepLeftoverProbes directly and its comment says why; the own-leftover test goes through EnsureCLIRemote. While there, EnsureCLIRemote runs ValidateGitDataRefArg on the ref before anything else, so a bad ref is refused as an argument and never reaches the probe's wording. Tests: TestCLIMirrorSweepsLeftoverProbes plants a leftover under this pid, one under a pid that has exited, one under a live pid, and a bd-ref-probe-backup, checks that exactly the first two are gone after EnsureCLIRemote, then replants the own-name leftover and checks that a move back to the default ref sweeps it too, so that half holds where pid liveness cannot be read, with the dolt CLI and skipped without; TestLeftoverProbe covers the name shapes, and TestCLIMirrorSweepReportsUnreadableState, TestCLIMirrorOwnLeftoverRemovalFailureIsReported, and TestCLIMirrorInvalidRefIsRefusedUnwrapped the three wordings, without a dolt binary; TestCLIMirrorInvalidRefIsRefusedBeforeProbe runs the ref check over an existing remote, the case where it used to come back worded as the server's refusal, with the dolt CLI. The dead-pid assertions are gated on the platform's liveness check, so the suite holds on Windows too, where only the own name is swept.

Notes.

  • Fail-closed repo_state.json read: agreed, and the body now carries it as the one deliberate exception to "nothing changes for a remote without a ref", with your reasoning.
  • PersistedRemotes failing whole on one malformed params: left as is. The callers have a binary contract, "remotes" or "could not tell", and ignore the slice on error; a partial slice beside an error would need a new contract and a caller that uses it, and dolt only writes objects there. Go's error already names the field path, .remotes.params.
  • Reading the ref from dolt remote -v: left as is, and the body now records the choice. The listing's rendering of parameters is not a documented format, JSON with internal spaces, so a Fields split breaks on it; and at GH#2118 cold start the proxied listing is empty, which is exactly the case where the state file is the only source. You found no divergence guard 1 misses, and neither did I.
  • bd dolt remote reset-data ref-blind: feat(dolt): --ref on bd dolt remote add and bd bootstrap, sync.remote-ref #6525 makes lsRemoteDoltDataRefs take the remote's ref and parseDoltDataRefs keep only the exact data ref and the info ref out of what git ls-remote returns, so a reset on refs/dolt/units/<key> deletes that ref and never refs/dolt/data. One thing worth knowing when you read it there: the info branch, refs/heads/__dolt_remote_info__, is one marker file per repository, written by every push with the ref of that push, dolt's pushInfoBranch with last writer wins, so deleting it beside a custom data ref costs nothing; the next push of any database in the repository recreates it.
  • U+00A0: the mismatch was the doc comment, and that is what changed. I first widened the check to Unicode whitespace, then checked the premise: git check-ref-format 'refs/heads/issue<U+00A0>data' succeeds and dolt 2.3.2 takes the same ref on remote add --ref, so that widening would have rejected a valid ref on the CLI-mirror path while the SQL path accepted it. The check stays a space, an ASCII control character, DEL, and the leading dash: the first three are bytes git's ref rules refuse, the dash is so the value can never be mistaken for an option; dolt itself accepts all four through exec, so this is bd policy at the argv boundary, not a ref-name check. The comment now says exactly that, and TestValidateGitDataRefArg pins both sides: the three non-ASCII spaces pass, the ASCII cases and the dash are refused.

Verification on the new head: BEADS_TEST_ENV_RUN_DOLT=1 ./scripts/test.sh ./internal/storage/ ./internal/storage/versioncontrolops/... ./internal/storage/doltutil/... ./internal/storage/domain/... and the internal/storage/dolt and embeddeddolt runs from the body, all ok with dolt 2.3.2 on PATH; go vet for darwin, windows, and js/wasm on doltutil, golangci-lint on ./internal/storage/... 0 issues; make test ok apart from one failure in scripts/repro-dolt-prod-timeouts, untouched by this PR and green when run alone.

@sdghbtn

sdghbtn commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review of this push, on the delta since 0c0a57d45: three rounds under each of two models, six in all, each round a fresh context in a throwaway worktree of the branch with the diff, the body, the draft reply, and the earlier rounds' dispositions.

Findings: 25 in total, 0 blocker, 4 major, the rest minor or question. 24 accepted and fixed on the branch; 1 question answered without a change, the remedy's quoting under cmd.exe, which follows the 72fc86cf1 precedent. What the rounds drove: the probe sweep runs before every change to the mirror and admits only the probe's exact name shape, with another run's cleanup described as best-effort rather than guaranteed; the PersistedRemotes read error and a failed own-name removal come back with their own wording, never as the server refusing parameters; the ref is validated at the top of EnsureCLIRemote, pinned by a no-dolt test that fails when that check is removed and by a dolt-backed test over an existing remote; the dead-pid assertions follow the platform's liveness check, and Windows gets the OpenProcess check internal/doltserver already uses, cross-compiled here and not run; the argument check's rationale corrected to what it is, bd policy at the argv boundary, since dolt accepts every one of those bytes through exec; the cold-start remedy built as one shell word.

Verification on this head, dolt 2.3.2 on PATH: the doltutil suite with the CLI tests, the internal/storage/dolt and internal/storage remote and mirror tests, go vet for linux, darwin, windows, and js/wasm, golangci-lint 0 issues on darwin and windows, make test with one unrelated failure in scripts/repro-dolt-prod-timeouts that passes alone and that neither commit touches.

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-read at efe18a802 on main at 7ccb961f7. The rebase is clean by range-diff against f56632adc..0c0a57d45: every changed hunk is one of the two asks or the whitespace note, and nothing from main's other store.go / version_control.go hunks leaks into the PR's delta.

Ran on this head, all ok: go build ./...; go vet ./internal/storage/doltutil/ for darwin, linux, windows and js/wasm; with BEADS_TEST_ENV_RUN_DOLT=1 on dolt 2.3.1: ./internal/storage/doltutil/... (the sweep, leftover-shape, unreadable-state, own-removal-failure and both invalid-ref tests included) and -run 'TestColdStart|TestCLIMirror|TestPrepareCLIRoute' ./internal/storage/dolt/; -run TestRemoteGitDataRefOnEngine ./internal/storage/embeddeddolt/ on the real engine.

1. The remedy now names a command that exists and repairs the condition. Checked the premises rather than the wording: bd sql is cobra.ExactArgs(1), so the single-quoted statement is one argument; a CALL goes through ExecContext past CheckReadonly, and the guard fires on the push/pull route decision, not on open, so the store bd sql needs is reachable in the GH#2118 window. DOLT_REMOTE takes --ref in the pinned engine (dprocedures/dolt_remote.go:110), and on a 2.3.1 repo CALL DOLT_REMOTE('add', '--ref', 'refs/heads/issue-data', 'origin', 'git+https://example.com/o''hare/repo.git') records {"git_ref": "refs/heads/issue-data"} in repo_state.json and shows it in dolt remote -v; the second call returns remote already exists, as your reply says. The double-quoted literal form the remedy prints works the same way, with a " and a \ in the URL landing intact (dolt percent-encodes them in the stored URL, which is dolt's normalisation and not this PR's concern). TestColdStartReaddRemedyQuotes pins the shell word.

2. The sweep does what the body now says: the exact bd-ref-probe-<pid> shape in canonical decimal, a dead pid only where the platform can tell (ESRCH alone on unix; OpenProcess plus a zero wait on Windows, WAIT_OBJECT_0 read as exited, access-denied as alive), the own name always, and a failed removal of another run's leftover ignored while a failed own-name removal is returned in its own words. TestCLIMirrorSweepsLeftoverProbes plants all four shapes and checks exactly the own and dead ones go, on the dolt CLI. The PersistedRemotes error is no longer swallowed; TestCLIMirrorSweepReportsUnreadableState covers it. ValidateGitDataRefArg at the top of EnsureCLIRemote closes the mis-wording over an existing remote, and the U+00A0 question is settled the right way round: the comment changed, the check did not, and the test pins both sides.

The notes are disposed as you describe; reset-data I'll read on #6525 as planned.

Approving. CI has not run on this PR (fork workflow awaiting approval), so the Windows path is vet-only from here as well; nothing in it is reached on the two platforms the tests ran on.

@gascityinc-sarah gascityinc-sarah Bot added kind/feature New user-facing capability priority/p3 Backlog: useful but not currently planned labels Sep 26, 2026
@gascityinc-sarah

Copy link
Copy Markdown

Triage: kind/feature · priority/p3 — Storage-layer groundwork so a later change can expose a ref option without reaching across layers; it carries the git data ref of a git-backed Dolt remote and changes no user-facing behavior on its...
Auto-triaged by @gascityinc-sarah. Reply or relabel if this is off.

@gascityinc-sarah gascityinc-sarah Bot removed the status/needs-triage Awaiting initial PM triage label Sep 26, 2026
@julianknutsen julianknutsen added the status/needs-review-auto Request automated PR review workflow label Sep 26, 2026
@julianknutsen

Copy link
Copy Markdown
Collaborator

This PR has been added to the review queue.

Track progress: https://factory.gascity.com/city/maintainer-city/runs/gcg--9223372036854769982

@julianknutsen julianknutsen added status/reviewing PR review workflow is running and removed status/needs-review-auto Request automated PR review workflow labels Sep 26, 2026
sdghbtn and others added 2 commits September 26, 2026 18:40
Dolt lets a git-backed remote keep its data on a ref other than
refs/dolt/data (the git_ref remote parameter, --ref on dolt remote add,
dolt clone, DOLT_REMOTE, and DOLT_CLONE). The storage layer never passed
it through, so a host that only accepts pushes under refs/heads/, or one
repository holding several databases on refs such as
refs/dolt/units/<key>, could not be used. Part of gastownhall#6261; the commands
and the config key follow in a stacked PR.

- storage.RemoteInfo gains Ref (empty = refs/dolt/data);
  RemoteRefsMatch, EffectiveGitDataRef, and GitRefFromParamsJSON give
  every package one reading of it, pinned against dolt's constants.
- RemoteStore.AddRemoteWithRef on both stores, and CreateRemoteWithRef
  on the proxied-server remote use case, pass a non-empty ref as
  DOLT_REMOTE's --ref. AddRemote is unchanged; UpdateRemote keeps the
  ref through replacement and restore.
- versioncontrolops.DoltCloneWithRef and dolt.BootstrapFromRemoteWithDBRef
  pass the ref to DOLT_CLONE and to the dolt clone argv.
- ListRemotes reads the ref back from dolt_remotes.params (git_ref) in
  both remote repositories; PersistedRemotes reads it from
  repo_state.json. Malformed params is an error, never a silent default.
- The server-mode CLI mirror is compared on URL and ref and
  re-materialized with dolt remote add --ref when the ref differs. Where
  the CLI is proxied to the running server: the check reads the server's
  own repo_state.json when the listing is empty, the cold-start fallback
  re-adds a ref remote over SQL (which accepts --ref) so the server knows
  it, and a throwaway probe remote proves --ref works before an existing
  mirror remote is removed; probe remotes left by runs no longer alive
  are swept before any change to the mirror. For a server this bd does
  not own, the route stops and names the DOLT_REMOTE call to run
  against that server through bd sql, as one shell-quoted word.
- A ref is checked for a space, an ASCII control character, DEL, and a
  leading dash before it becomes a dolt argument; no other ref-name rule
  applies at this layer.
- Tests carry both ref shapes; the embedded-engine test proves add, list,
  push, and clone on the real engine; sqlmock tests cover the route
  decider and the cold-start re-add at the store seam; the dolt-CLI tests
  skip without a dolt binary.
…astownhall#6521 review)

Maintainer review fixup for PR gastownhall#6521. The review found no blocker or major;
this is the non-gating minor/nit set plus the test pins the synthesis named as
the residual defects.

- One reader for git_ref. storage.GitRefFromParams is now the single decoder
  for both planes (the SQL params column through GitRefFromParamsJSON, and
  repo_state.json through doltutil.PersistedRemotes), and a git_ref that is
  present but not a string is an error naming the remote instead of silently
  the default ref. Reading it as the default is how a mirror pinned to a custom
  ref would be kept on refs/dolt/data -- the wrong-ref push this PR closes,
  reached from the opposite direction.

- CreateRemoteWithRef checks the ref against what the dolt argv boundary will
  accept, so the recordable ref set is no longer wider than the routable one
  and a bad ref is refused at add time rather than diagnosed at push time.
  UpdateRemote deliberately does not re-check: it carries and restores refs
  that are already recorded and must stay able to put one back.

- BootstrapFromRemoteWithDBRef validates the ref before the already-exists
  early return, so the same --ref is accepted or refused the same way on a
  fresh host and on a bootstrapped one.

- sweepLeftoverProbes warns when it cannot remove another run's leftover probe
  remote, following PersistedRemoteInfos' precedent: the remote stays visible
  in `bd remote list` until some run succeeds, and nothing ever said why.

- EnsureCLIRemote's doc comment records that escalating a state-file read error
  is intentional for default-ref remotes too, and
  TestCLIMirrorUnreadableStateIsAnError now pins the ref == "" direction that a
  later "helpful" relaxation would gate away. The suggested URL-match
  short-circuit is deliberately NOT applied: when the state file is unreadable
  currentRef is unknown, so returning nil on a URL match alone would route a
  push onto whatever ref the mirror is secretly pinned to.

- AddCLIRemote's doc comment records that it is kept deliberately as the
  default-ref spelling now that EnsureCLIRemote uses AddCLIRemoteWithRef.

Not applied: widening probeCLIRemoteRef to `ref != "" || currentRef != ""` and
probing the restore shape. On a proxied server that turns a working custom-ref
-> default-ref transition into a hard failure (the target add is plain and
succeeds today, while probing the restore shape would be refused), and in the
one window that survives calibration -- a non-proxied plain-add failure from
disk/lock contention -- a capability probe predicts nothing about it.

Each new pin was mutation-proved: reverting the fix it guards reds that pin,
and only that pin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@julianknutsen
julianknutsen force-pushed the feat/dolt-remote-git-ref-storage branch from efe18a8 to 0801749 Compare September 26, 2026 18:44

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

R2 review at 0801749 (the maintainer fixup over my R1 approve at efe18a8). Approve.

What changed since R1: rebased onto main c25059a (range-diff: the original commit is =) plus one commit. Both git_ref readers, storage.GitRefFromParamsJSON over the dolt_remotes params column and doltutil.PersistedRemotes over repo_state.json, used to do v, _ := params["git_ref"].(string), so a git_ref that was present but not a string was silently read as the default refs/dolt/data, which is the wrong-ref push this PR exists to close. They now share one reader, storage.GitRefFromParams: missing key or JSON null still means the default; any other non-string value is an error, and PersistedRemotes names the remote in it. The commit also checks the ref in CreateRemoteWithRef before anything is recorded, moves the ref check in BootstrapFromRemoteWithDBRef ahead of the "database already exists" early return, and makes the probe sweep warn instead of silently ignoring a failed removal of another run's leftover probe. Every git_ref read in non-test code now goes through the shared reader; the two callers that still swallow its error (persistedCLIRemote, PersistedRemoteInfos) land on the no-push side. 29 files, all under internal/storage/; merge-tree against main clean; the main delta since the merge-base touches none of them.

Ran here (go1.27.1, dolt 2.3.5), at 0801749: go build ./...; go vet on the touched storage packages; gofmt clean. go test over ./internal/storage/, domain/..., versioncontrolops/..., doltutil/... with BEADS_TEST_ENV_RUN_DOLT=1: all ok, 0 SKIPs. ./internal/storage/dolt/ -run 'TestColdStart|TestCLIMirror|TestPrepareCLIRoute|TestBootstrap|TestDoltCloneArgs': ok, 37 run, 0 SKIPs. Embedded tier (BEADS_TEST_EMBEDDED_DOLT=1 -tags cgo): TestRemoteGitDataRefOnEngine, TestGitRefPinsMatchDolt PASS.

Mutations (each reverted, tree clean):

  1. Shared reader returns "", nil on a non-string: remote_ref_test.go:70 reds for array, number, bool and object (want an error, not the default ref), TestGitRefFromParamsJSONRefusesNonStringRef reds, and persisted_remotes_test.go:104 reds (PersistedRemotes with a non-string git_ref = [{Name:unit … Ref:}], nil; want an error).
  2. Only the on-disk reader falls back to the silent assert: the doltutil pin reds while the storage package stays green, so each reader has its own pin.
  3. CreateRemoteWithRef check removed: remote_test.go:131 "should be refused" for -x, space, tab; :135 "a refused CreateRemoteWithRef reached the repository".
  4. Bootstrap ref check moved back behind the early return: bootstrap_test.go:174 reds.
  5. EnsureCLIRemote escalates only when a ref was asked for: cli_mirror_ref_test.go:390 reds.

Not blocking, for the record: the pin in (5) goes red on message wording (the mutated code still fails, later, in the probe sweep, with a message lacking "recorded ref"); the case that matters, URL match + empty ref + unreadable state file, is not in the fixture. The new sweep warning has no test. One non-string git_ref now fails PersistedRemotes for every remote in that directory, the same fail-closed trade-off noted last round; Dolt only ever writes a string there.

@julianknutsen

Copy link
Copy Markdown
Collaborator

Maintainer Adoption Review

Thanks for the contribution, @sdghbtn! This PR makes a git-backed Dolt remote's git data ref a first-class part of the remote record — carried through the SQL plane (dolt_remotes.params), the CLI mirror (repo_state.json), clone, and bootstrap. That closes a real hazard: because no plane carried the ref at all, a remote pinned to a custom ref could be pushed onto refs/dolt/data instead, and that failure is silent — the remote looks healthy and syncs nothing.

This PR was reviewed and adopted with maintainer fixes pushed directly to the PR branch.

Original PR Review

Decision: approve.

Specific gaps fixed:

  • The PR's premise held under independent check. Both reviewers verified against the pre-change tree that nothing already carried the ref: versioncontrolops.ListRemotes and db.ListRemotes selected name, url only, doltutil.PersistedRemotes decoded url only, and neither AddCLIRemote nor DoltStore.AddRemote emitted --ref. No pre-existing handler, silent default, or fallback to collide with.
  • The new Ref really is carried on every routing plane — no silent no-op. DoltStore.ListRemotes / EmbeddedDoltStore.ListRemotes → versioncontrolops.ListRemotes, and PersistedRemoteInfos → doltutil.PersistedRemotes, are both updated, and all four ensureMatchingCLIRemote / hasMatchingCLIRemote call sites pass r.Ref.
  • dolt_remotes.params really scans on the embedded engine. An early sql.NullString concern was refuted by running your TestRemoteGitDataRefOnEngine and TestGitRefPinsMatchDolt against a real engine.
  • The one major that was filed got adjudicated down, and its prescribed fix deliberately not applied. The suggestion was for EnsureCLIRemote to short-circuit to nil on a URL match when repo_state.json is unreadable. Applying it would have reintroduced exactly the wrong-ref push this PR closes: with the state file unreadable, currentRef is unknown, so a URL-match short-circuit can route a push onto whatever ref the mirror is secretly pinned to. Your fail-closed behaviour is the right call — the only residual was that nothing pinned it.

Review findings addressed:

  • A non-string git_ref read silently as the default ref — the opposite-direction hole in the PR's own fail-closed invariant. storage.GitRefFromParams is now the single decoder for both planes, and a present-but-non-string git_ref is an error naming the remote (0801749).
  • The recordable ref set was wider than the routable ref set — a ref with a space or a leading dash was accepted by CreateRemoteWithRef and refused only later at the argv boundary, so the remote was creatable but permanently un-pushable, diagnosed at push time instead of add time. CreateRemoteWithRef now validates at the write seam; UpdateRemote deliberately does not re-check, so it can still put back an already-recorded ref (0801749).
  • BootstrapFromRemoteWithDBRef validated the ref after the already-exists early return — so --ref "-x" was accepted on a bootstrapped host and refused on a fresh one, i.e. the same misconfiguration only surfaced on a new machine. Validation now runs first (0801749).
  • sweepLeftoverProbes silently swallowed failures to remove another run's leftover probe remote — it now warns, following PersistedRemoteInfos' precedent, so a stuck bd-ref-probe-<pid> in bd remote list finally has an explanation (0801749).
  • The fail-closed state-file escalation had no pin for the default-ref direction — TestCLIMirrorUnreadableStateIsAnError now covers ref == "", and EnsureCLIRemote's doc comment records that the escalation is intentional there too, so a later "helpful" relaxation trips a test instead of shipping (0801749).
  • AddCLIRemote had no remaining production caller — kept deliberately as the default-ref spelling now that EnsureCLIRemote uses AddCLIRemoteWithRef, and its doc comment now says so (0801749).

Each new pin was mutation-proved: reverting the fix it guards reds that pin, and only that pin.

Non-blocking follow-ups (shared for visibility — NOT completed in this PR):

  • bd-2dozu — Error Handling & Resilience, minor: probeCLIRemoteRef early-returns when the target ref is empty, so the currentRef != "" && ref == "" direction is never probed even though the rollback is the path that passes --ref. The prescribed widening was deliberately not applied here — on a proxied server it would turn a working custom-ref → default-ref transition into a hard failure, and in the one window that survives calibration (a non-proxied plain-add failure from disk or lock contention) a capability probe predicts nothing about it. Filed so the trade-off gets decided rather than assumed.
  • bd-xg5in — Open risk, minor: the proxied asymmetry this design rests on (a dolt sql-server refuses --ref through the proxied CLI, and the SQL re-add makes the server see the remote) is argued from a code walk plus GH#2118, not exercised end to end. Not merge-blocking: probe-before-delete means a wrong assumption fails loudly rather than pushing to the wrong ref, and the feature is dormant until a caller passes a non-empty ref.
  • bd-cbr9s — Open risk, nit: the ensure-after-SQL-re-add check trusts the served directory's repo_state.json to be flushed promptly, and strings.Contains(err, "already exists") in store.go is pinned to Dolt's error text. Both are couplings to Dolt implementation details we do not control; recorded together so one hardening pass can cover both.

Maintainer Changes

One maintainer fixup commit, 0801749 — fix(storage): make both git_ref readers fail closed; pin the edges (#6521 review) — 9 files changed, +224/−22. It applies the non-gating minor/nit set above and adds the test pins the review named as the residual defects; your commit is preserved with its original authorship. The branch was also rebased onto main at c25059ad before the push, and go build ./..., go vet ./internal/storage/..., plus the touched packages' tests were re-run green on that refreshed head. main has moved on since, so I re-ran the same gates against the branch rebased onto the current tip (93cfee90): the rebase is clean and the patch is byte-identical (same patch-id across both refreshes). The only file this PR shares with that drift is store.go, where #6554 added --user to the fetch/pull/push argv while this PR works on the remote add --ref argv through a separate exec site, so I ran #6554's transfer-argv tests alongside this PR's own — both green together. No further rebase is needed before the queue picks it up.

Final Review Status

Ready for the merge queue: final head 0801749 has passing required GitHub checks, and the merge-ready metadata is in place. I am marking this PR status/merge-ready so the merge queue can pick it up.

CI: https://github.com/gastownhall/beads/actions/runs/36263641540

Review Iterations

1 review pass, by two reviewers (Claude + Codex) who independently validated the PR's premise against the pre-change tree rather than taking the PR body's word for it. No blocker or major survived; the non-gating minor/nit set was applied as the single maintainer fixup above. One coverage caveat worth stating plainly: a third reviewer lane was skipped for this run, so cross-file coverage here is two-reviewer rather than three.


Adopted via /adopt-pr workflow. Original contributor commits preserved.

@julianknutsen julianknutsen added the status/merge-ready PR is ready for merge workflow label Sep 27, 2026
@julianknutsen julianknutsen added status/merge-queued Queued for deterministic PR-review merge and removed status/merge-ready PR is ready for merge workflow status/reviewing PR review workflow is running labels Sep 27, 2026
Resolve internal/storage/dolt/credentials.go: main (gastownhall#6454 execenv work) replaced
the nested os.Environ()/strings.HasPrefix scan with execenv.ContainsKeyWithPrefix,
which prefix-matches the env KEY only and applies Windows key identity. This PR
independently threads the git data ref through ensureMatchingCLIRemote. The two
changes are orthogonal, so keep both: main scan structure + this PR three-arg call.
The auto-merged store.go already carries the three-arg definition, so the
two-arg arm would not compile.
@julianknutsen
julianknutsen merged commit 3efcd35 into gastownhall:main Sep 27, 2026
122 checks passed
@julianknutsen julianknutsen removed the status/merge-queued Queued for deterministic PR-review merge label Sep 27, 2026
julianknutsen added a commit to ecuthiell/beads that referenced this pull request Sep 28, 2026
…6464, round 2)

main advanced two commits while the first merge was being resolved: gastownhall#6521
(carry the git data ref of a git-backed Dolt remote) and gastownhall#6461 (retain explicit
null config suppression). gastownhall#6521 is disjoint from this PR and auto-merged. gastownhall#6461
lands on the same routing boundary this PR hardens, so all three conflicts are
two descriptions of one boundary rather than competing behavior.

Resolutions:

- cmd/bd/worktree_cmd.go: union. Keep main's new first paragraph, which records
  that startup config discovery and the already-selected .beads probes share the
  boundary while the generic internal/beads and internal/git/gitdir.go planes do
  not (bd-p4che) -- that is new information from gastownhall#6461. Keep this PR's reviewed
  three-bullet clause split over main's one-sentence version: it says everything
  main's sentence said, preserves main's "(init's role probe records the same
  trade-off)" parenthetical, and additionally distinguishes GIT_CEILING_DIRECTORIES
  (a stop, so clearing widens the walk) from GIT_DISCOVERY_ACROSS_FILESYSTEM
  (permit-only, so clearing narrows it). Fold gastownhall#6461's suppression exception into
  the GIT_CONFIG* bullet, where it belongs: ClearRouting matches GIT_CONFIG by
  prefix but keeps GIT_CONFIG_NOSYSTEM and the /dev/null spellings, which can
  blind a read but cannot redirect one.
- cmd/bd/config.go: keep this PR's comment naming the four verbs
  (get/set/unset/set-many). It is the narrower and still-true form, and it is the
  remedy a review finding asked for; main's "bd config's own reads and writes"
  phrasing is the superseded generalization.
- internal/storage/domain/git/git_test.go: superset. Keep every assertion from
  both sides -- main's require.Error message, ErrorContains(".gitconfig") and the
  inherited SetConfig refusal, plus this PR's stricter exec.ExitError/exit-128
  pin. The two sides assert the same failure at different strengths, so neither
  contract is dropped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hukadev added a commit to hukadev/beads that referenced this pull request Sep 28, 2026
…light-terminal-glamour-style

* gastownhall/main: (157 commits)
  fix(hooks): identify the index context behind tracked-file refusals (gastownhall#6457)
  fix(gitenv): retain explicit null config suppression (gastownhall#6461)
  feat(storage): carry the git data ref of a git-backed Dolt remote (gastownhall#6521)
  fix(init): write embedded excludes in the selected project (gastownhall#6462)
  fix(doctor): avoid a leading separator in empty gitignore (gastownhall#6421)
  fix(notion): keep notion.token out of the Dolt database (gastownhall#6676) (gastownhall#6724)
  test(bazel): run every Bazel test against a hermetic, pinned dolt CLI (gastownhall#6853)
  fix(init): bootstrap Git in the selected project directory (gastownhall#6459)
  fix(doctor): escape database identifiers instead of rejecting them (gastownhall#6412)
  test(bazel): hermetic test env + inject bd binary into subprocess tests (gastownhall#6843)
  fix(doctor): report stealth exclude read failures without repair advice (gastownhall#6427)
  refactor(git): resolve hooks paths in an explicit read-only context (gastownhall#6436)
  fix(prime): honor shared-server mode consistently across bd prime's store resolution paths (gastownhall#6589)
  test(bazel): runfiles helpers + declared inputs for scan-guard tests (gastownhall#6828)
  fix(test): point tx-minting guard at commitWriteTx (gastownhall#6833)
  docs(recovery): init-safety re-clone playbook covers damaged-store and fresh-clone gotchas (gastownhall#6431)
  build(bazel): side-by-side Bazel scaffold + policy tests (gastownhall#6798)
  Accept uutils coreutils timeout in generated hooks, and check the shims against real implementations (gastownhall#6561)
  fix(list): expose comment bodies in bd list/show --json (be-73x) (gastownhall#6473)
  fix: dep add stores an unresolved target as a bare external ref, then readers drop it (gastownhall#6571)
  ...

; Conflicts:
;	CHANGELOG.md
;	docs/recovery/init-safety.md
rjgeng pushed a commit to rjgeng/beads that referenced this pull request Nov 3, 2026
…it repository (gastownhall#7902)

bd dolt remote reset-data classified a file:// remote by what is on disk.
Dolt makes a file URL a git remote only when it ends in .git (stored as
git+file://); any other file:// URL is a native Dolt file store, which
Dolt writes among the files of a git repository at that path. Once Dolt
had pushed into a checkout, reset-data took the directory for a file
store and cleared it, .git and every working file included, then
reported success. For a bare repository it deleted no Dolt data refs,
force-pushed, and reported a reset that left every old table file in
place.

reset-data now refuses a file:// remote (or bare path) whose directory
is a git repository: any .git entry (a directory, as in a checkout, or a
file, as in a linked worktree or submodule), or a bare repository. It
gives the replace-the-remote hint it already gives a cloud remote, and
the help text says so.

Tests: TestClassifyResetDataRemote gains gastownhall#7902's legs (a bare repository
with and without a Dolt store, a checkout holding a store by file:// URL
and by bare path, a linked worktree whose .git is a file, a checkout
without a store); with the old classification the checkout and worktree
legs holding a store read as file stores (= 1, want 4), the bare legs as
git-backed (= 0, want 4), and the checkout without a store as an error.
TestEmbeddedDoltRemoteResetData_FileRemoteInGitRepoRefused pushes to a
file:// remote in a bare repository and in a checkout, then checks that
reset-data --yes refuses each and leaves the directory, .git, README.md
and the git history intact. Run against a bd built from hotfix/1.3.2
(BEADS_TEST_BD_BINARY) it fails: reset-data exits 0.

Backport of gastownhall#7902 (c153e3b).

[hotfix/1.3.2 backport] Ports item 4 of gastownhall#7902 only (the file:// remote in
a git repository); this line predates the endpoint-ownership chain the
rest of the PR builds on.

- Ported from gastownhall#7902: resetDataFileInGitRepo, isGitRepoDir, the
  classifyResetDataRemote doc comment and refusal, the RunE refusal
  message, the help-text line, and the TestClassifyResetDataRemote
  table. The two existing legs that sent a file:// URL or bare path to a
  bare repository down the git-backed route now expect the refusal, as
  on main.
- The refusal's hint prints the remote name as this line's cloud-remote
  refusal does; main quotes it through internal/remedy, which is not on
  this line.
- Added for this line: TestEmbeddedDoltRemoteResetData_FileRemoteInGitRepoRefused,
  the bare-repository and checkout refusal legs of gastownhall#7902's server-mode
  TestE2E_ResetDataThroughOwnServerKeepsTheRemotesData, run in embedded
  mode (the classification runs before any push, in either mode; that
  E2E itself needs the server route, which is not ported). Its TestEmbedded
  prefix puts it in CI's embedded cmd shards.
- Not ported:
  - Item 1 (cliDirServesServer requiring doltserver.ServesDatabaseDirOnPort,
    the databaseLocked probe, testutil.HoldDoltDatabaseLock, and the
    storage-boundary exception in AGENTS.md and engdocs/PROJECT_CHARTER.md):
    it narrows gastownhall#7870/gastownhall#7893's host-ownership predicate and leftover-remote
    refusal, which build on gastownhall#7840/gastownhall#7841/gastownhall#7874; none of them is on this
    line, where hasCLIDatabase checks only that .beads/dolt/<db>/.dolt
    exists.
  - Item 2 (the leftover refusal's wording): gastownhall#7893's
    remoteOnlyInLeftoverDatabaseError does not exist on this line.
  - Item 3 (reset-data's server route: a temporary remote and ref, a
    leased move of refs/dolt/data, and a restart of bd's server): it needs
    AddRemoteWithRef and RemoteInfo.Ref (gastownhall#6521), doltserver.Restart,
    DoltStore.ManagedServerDir and internal/remedy, none of which is on
    this line. Embedded and server mode keep delete-then-force-push.
  - Item 5 (docs/recovery/history-squash.md): the published runbook is
    served from main, which already carries the change.
  - resetDataGitURL's scp-style and port handling, resetDataProgressf
    (--json-only stdout), CHANGELOG.md (release prep writes it), the
    BUILD.bazel files and tools/nogo list (no Bazel build on this line),
    and the command-tree golden (absent on this line).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature New user-facing capability priority/p3 Backlog: useful but not currently planned

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants