Skip to content

feat(dolt): --ref on bd dolt remote add and bd bootstrap, sync.remote-ref - #6525

Open
sdghbtn wants to merge 5 commits into
gastownhall:mainfrom
sdghbtn:feat/dolt-remote-git-ref-cmd
Open

sdghbtn wants to merge 5 commits into
gastownhall:mainfrom
sdghbtn:feat/dolt-remote-git-ref-cmd

Conversation

@sdghbtn

@sdghbtn sdghbtn commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

What

The command layer for the storage primitives of #6521: a git-backed Dolt remote can be added, bootstrapped, listed, and reset on a git ref other than refs/dolt/data.

  • bd dolt remote add <name> <url> --ref <ref>. Only a full, well-formed ref name is accepted: git's check-ref-format rules (so no refs/heads/* pattern that a later git ls-remote would expand), and not dolt's effective remote info branch (refs/heads/ plus __dolt_remote_info__ or whatever DOLT_REMOTE_INFO_BRANCH names; dolt adds that prefix unconditionally), which dolt overwrites after every push. A bare branch name, a typo such as ref/heads/x, or an empty value is an error naming the rule, checked in cobra's argument phase, before the root pre-run opens any storage. The same rules run in bd config set sync.remote-ref (config.ValidateGitDataRef), and a hand-edited committed value that fails them is ignored with one warning instead of reaching dolt or git. A ref on a URL dolt does not treat as git-backed is refused by bd before dolt sees it, naming the rule and the remedy; git-backed follows dolt's own NormalizeGitRemoteUrl: a git+ scheme, or a file/http/https/ssh URL, an scp-style host:path, or an absolute local path ending in .git. A ref under refs/heads/ or refs/tags/ is checked against the repository with one git ls-remote --symref before the remote is created or replaced, never on a re-add that changes nothing: the repository's default branch is refused outright, and a ref that already exists is taken only with --yes or a confirmed prompt, because dolt's first push replaces the ref's tip with Dolt storage. A probe that fails says nothing about the ref, the default branch included, so with a terminal it warns and does not block, and without one it is a refusal, --yes or not, since nobody reads the warning on an unattended run. The same guard runs on every path that creates or moves origin on the configured ref without having proved what the ref holds (bd bootstrap wires a remote only after its probe found the ref absent or a clone proved Dolt data on it, so it needs none): when bd dolt push adopts the git origin (a prompt with a terminal, as on remote add); when bd config apply adds a fresh origin on federation drift or moves it to a new URL (checked against the new repository, since the ref was admitted against the old one), and in its dry run, which reports the refusal the live run would make; when proxied bd init configures origin; and whenever direct bd init wires a fresh database to the configured remote on the ref rather than bootstrapping from it (the clone of the ref found no Dolt data, the probe found the ref absent, or the probe failed; the origin probe reports a branch holding source as found, and a clone that succeeded is proof the ref holds Dolt data and is not re-checked). The paths that cannot ask (bd config apply, proxied bd init, and bd init in non-interactive mode: --non-interactive, BD_NON_INTERACTIVE=1, CI=true, or no terminal) refuse the default branch, an existing ref, and a ref the probe could not read; the refusal names the remedy of its path for taking an existing ref on purpose, shell-quoted and credential-free: bd dolt remote add origin <url> --ref <ref> --yes where the workspace owns its store (with --allow-git-origin when the URL is the git origin), a bd sql call of DOLT_REMOTE over a proxied server, where remote add is refused, and for a move by bd config apply, whose ref comes from the remote record rather than the key, a re-add of origin with --yes or with another --ref. A remote divergence the user confirmed during bd init (the destroy-token flow) counts as consent to take an existing ref; the default branch is refused either way, and so is a ref the probe cannot read whenever nobody is watching, a pty under --non-interactive included. bd config apply reports the refusal as an error result, in its dry run too, which runs the same read-only probe and exits non-zero where the live run would instead of promising an add; proxied bd init fails as it does when the remote cannot be created; direct bd init leaves origin unconfigured, says why, names the bd dolt remote add origin <url> --ref <ref> that finishes the job, and goes on, as it does when the add itself fails. Init does not run twice, and the next bd dolt push adopts the git origin rather than sync.remote, which is the right remote only when the two are the same repository, so the remedy is the add and never a change to the key alone; adoption preferring the git origin over a configured sync.remote is main's behavior and a follow-up. --ref refs/dolt/data is the default.

  • A ref only exists on a git-backed remote, so a ref beside a sync.remote that is not one (file://, dolthub://, a remotesapi endpoint) is handled before dolt sees it: an explicit bd bootstrap --ref there is refused naming the URL, and a configured key is dropped from the plan, from the bd init clone and origin (embedded, server, proxied-server), and from bd config apply, with one warning naming the key and how to clear it; bd dolt remote add origin <url> without --ref is the one path that refuses instead of dropping, since the add is explicit and the user can name the ref. bd config set sync.remote prints the same hint when a key is set and the new URL cannot carry it, judged in the routed form bootstrap and init store: a forge URL without .git gets no hint, since its git+ form carries the ref. A git origin that is a local path without .git is spelled git+file:// when a ref is in play, by bootstrap, init, and push-time adoption alike, since a scheme-less, host-less path is an origin shape dolt does not read as a git repository: it goes to the DoltHub remotesapi host. A file:// or relative-path origin without .git is not respelled; the ref is dropped there with the default-ref warning, and the plan's reason says the clone looks at the default ref. For a forge URL, which bd init and bd bootstrap route to its git+ form before handing it to dolt, the ref is decided against that routed form: https://gitlab.com/org/repo is not git-backed as written, its git+https:// form is. bd config apply keeps origin's ref when it recreates the remote on a URL change, and on the restore after a failed add.

  • sync.remote-ref in .beads/config.yaml (a YAML-only key beside sync.remote), written when origin is added with a ref, read when origin is re-added without --ref and when bd dolt push adopts the git origin as the Dolt remote, where a git origin that is not git-backed for dolt, such as a bare path, is adopted on the default ref with a warning, and cleared by --ref refs/dolt/data on origin and by bd bootstrap --ref refs/dolt/data for the workspace it bootstraps. bd dolt remote remove origin clears sync.remote and keeps the key: it is the workspace's statement of where the data lives, so remove followed by add lands on the same ref instead of moving the data to the default with no prompt. Clearing writes the empty value, which means the default, rather than removing the key, because the two differ once a sync.remote-ref exists in the user-global config.yaml: an absent workspace key lets that value through, an empty one pins the default for every clone of the committed file. The ref that applies to a workspace is resolved in the order the configuration layers are merged, BD_SYNC_REMOTE_REF (empty counting as unset, as the merged reader has it), config.local.yaml, the workspace's config.yaml in either spelling, then the merged configuration, which at that point holds only the user-global value; the two workspace files are read directly rather than through the merged reader because that reader lets a flat sync.remote-ref: line in the user-global file shadow a nested workspace value (its lookup tries the joined path before the nested one), which would undo the pin. bd bootstrap, bd init, the origin probe, bd context on both routes, push-time adoption, bd config get sync.remote-ref, the sync.remote set hint, and the key clear that --ref refs/dolt/data on origin performs all use that one resolver, so bd config get reports the ref that governs the workspace and an explicit default pins it here whichever layer the ref came from; the bootstrap paths that persist or clear the key in a workspace they are creating read that one file alone, since their question is what the file says. TestResolveSyncRemoteRef_WorkspaceValueWinsOverUserGlobal pins absent, empty, set, flat, local-layer, and environment cases against both global spellings. bd config set sync.remote-ref "" (or refs/dolt/data) says it pins the default; bd config unset sync.remote-ref removes the key in either spelling since fix(config): dotted keys round-trip through SetYamlConfigInDir/UnsetYamlConfig (bd-zj95, 1.3.1) #6574 taught unset the nested form, and its hint says the next configured value is read. A committed file whose nested sync: block sits in a wholly indented root mapping is a shape generic unset does not reach on main either (fix(config): bd config unset reported success while the key stayed set #5760 taught it the indented flat key); it is not special-cased here. bd config set and unset show the side effect; bd context shows the ref beside the remote on both the direct and the proxied-server route. Every read of the key accepts the flat sync.remote-ref: spelling and the nested sync: block alike.

  • Re-adding a remote on a different ref changes where its data is pushed, so it asks first; without a terminal it is refused with both refs named unless --yes is given. A different URL on the same ref keeps today's confirmation.

  • bd bootstrap --ref <ref> overrides the key; the plan carries and prints the ref, sync_remote_ref in --json; the text plan names the remote and the ref on a local plan that wires the new database to a remote as well as on a clone, so the user confirms against what the run will do to a remote; all three clone modes pass it through the primitives, and the ref is persisted beside sync.remote once the plan has run. Since fix(bootstrap): probe git remotes for dolt data instead of rejecting; exit non-zero when bootstrap declines (#5743, #5663) #6037, bootstrap probes a sync.remote that looks like a code repository (a .git suffix, a well-known forge) for Dolt data instead of rejecting it, clones when the data is there, and otherwise falls through to local recovery or a fresh database wired to that remote; a probe failure is a blocked plan with a non-zero exit. With a ref, every step of that follows the ref: the probe of the sync.remote and of the git origin asks for the ref, the clone passes it, the messages and the blocked plan's git ls-remote hint name it, and a fresh database created because the ref holds no data yet gets origin on that ref with the key persisted, creating config.yaml and writing sync.remote into it, credential-free, when the remote came from the environment and none exists, so the first bd dolt push and the team's other clones agree on where the data lives. The key and the remote are recorded before origin is wired, and a failure to record them is an error naming the bd dolt remote add origin <url> --ref <ref> that finishes the job, with --allow-git-origin when the URL is the git origin, not a warning: the database then exists with no origin, which one command repairs, rather than an origin on a ref no other clone can learn of. An explicit --ref refs/dolt/data clears a configured key on every plan it runs, including one that wires no remote, and the note printed for such a plan says so instead of "no effect"; with no key configured it writes nothing, so it and a plain bd bootstrap leave the same files. A plan that recovers from a local backup or a git-tracked issues.jsonl after a failed probe wires no remote and carries no ref, so its JSON and the --ref note agree; a blocked plan carries the ref it could not verify only beside the remote its git ls-remote hint names; a Blocked plan with no such remote, the server-mode refuse plan of Fix: bd bootstrap can also silently create an empty server-mode database #6503 among them, carries no ref and no remote, so --json shows neither. A ref under refs/heads/ or refs/tags/ that the probe finds is reported as found, not as Dolt data, since only a ref under refs/dolt/ proves that; the dry run says so, and the clone reports what the ref holds. An explicit --ref that cannot apply to the configured sync.remote exits non-zero naming the URL; the bd init refusal for a remote that already holds Dolt history names the probed ref.

  • bd init reads the key for its clone and for the origin remote it configures, in embedded, server, and proxied-server modes; the proxied-server path now routes a URL dolt would not read as git-backed as written, a forge URL without .git, to its git+ form as direct init does (a change to what proxied init stores, not only to the ref; main stored the forge URL raw), stores a URL dolt already reads as git-backed as written, the scp form among them since bd's rewrite to git+ssh:// would turn its relative path absolute, and decides the ref against the stored form. bd's URL comparison does not fold the /./ dolt keeps in that stored scp form, so a re-add of the same scp URL in a proxied workspace reads as a URL change; main stored the scp form raw there too, so this is pre-existing, and a /./ fold in doltremote.Normalize is a follow-up beside fix(bootstrap): probe git remotes for dolt data instead of rejecting; exit non-zero when bootstrap declines (#5743, #5663) #6037's relative-path note.

  • bd dolt remote list shows the ref (text and JSON); bd dolt remote reset-data deletes and rebuilds the ref the remote record carries, never refs/dolt/data when a ref is set, and deletes only the exact data ref and the info ref out of what git ls-remote returns, so the other databases a repository holds on their own refs are untouched (the info ref is one DOLT_REMOTE.md marker branch that dolt force-pushes again on every push). The confirmation prompt and the refusal without a terminal name that ref beside the URL, since on such a repository the URL alone does not say which database is replaced; the JSON result carries ref as bd dolt remote add --json does, the recorded ref in canonical form when it is not the default and no key otherwise, with the refs actually deleted in deleted_refs; bd dolt remote list --json shows the same for every remote bd created, since bd canonicalizes before recording, and prints verbatim what dolt recorded for a remote added through dolt itself (an explicit --ref refs/dolt/data there lists as such). The history-squash runbook now points at reset-data instead of a hard-coded refs/dolt/data deletion.

  • The storage layer's cold-start remedy for a server this bd does not own stays as feat(storage): carry the git data ref of a git-backed Dolt remote #6521 words it, a DOLT_REMOTE("add", "--ref", ...) call through bd sql, rather than the bd dolt remote add --ref this PR adds: in that GH#2118 window ensureDoltRemoteGuarded falls back to the persisted remotes, finds the same URL and ref, and returns without a SQL add, so the command would not repair the condition the message describes.

  • Docs: the flag on both commands' help, sync.remote-ref in the configuration reference, one section in docs/getting-started/sync-setup.md covering both uses (a branch under refs/heads/ for a host that refuses other refs; several databases in one repository on their own refs), and a CHANGELOG entry. The branch shape gets a paragraph on what a data branch is exposed to that refs/dolt/data never was, workflow triggers and rulesets that cover all branches, and docs/reference/protected-branches.md carries the same qualifier. The generated docs/cli-reference/ pages follow the release pin and are untouched.

  • bd dolt remote reset-data commits pending working-set changes before it removes anything on the remote. Since fix(embeddeddolt): commit pending changes before push (#5433) #5795 the embedded store commits pending changes inside every push, so the command's delete-then-push order could remove the remote's data ref and then fail on the commit (a constraint violation in the working set, for one), leaving a remote no clone can bootstrap from; that held for refs/dolt/data on main, not only for a custom ref. The embedded store now commits first and a failing commit returns with the remote untouched and a hint naming bd dolt commit. Server mode, whose working set is shared, keeps refusing a dirty working set and checks again after the confirmation prompt, because another session can write while the prompt waits. The prompt's sentence about pending changes follows the store mode, and the help text names both behaviors.

  • After the first review: the target guard runs on every path that wired origin on the configured ref without it (the first bullet describes where and how each path refuses); the guard's messages, the probe's own error (git's first stderr line, credential-scrubbed, as the bootstrap probe reports it), the push-time adoption warning and the adoption announcement under --yes, init's no-ref confirmation and its probe-failure note print the URL through redactRemoteURL, the adoption URL being the git origin with whatever userinfo CI put in it, and redactRemoteURL itself now drops the userinfo of a scheme-less user:token@host/repo.git, which dolt reads as https, and spells the result https://, since the userinfo was what kept a dotted host before the first colon (host.example:443/repo.git) from reading as an scp address, for dolt and git alike, and bootstrap persists the redacted URL into sync.remote and the remedies paste it (the url fields of remote list --json and reset-data --json, and the other messages main already printed, are as they were); isGitBackedDoltRemoteURL falls through from a colon that makes no scp host to dolt's path rules, so a Windows drive-letter or UNC path ending in .git is git-backed on Windows (and /srv/le:dgers.git everywhere) instead of being refused for --ref on a remote dolt treats as git-backed, for the URL as given to remote add; a git origin still goes through doltremote.Normalize first, which spells a drive-letter path git+C:\..., a pre-existing shape neither bd nor dolt reads as git-backed. The Windows lanes build and smoke-test rather than run this package, so the Windows expectations in TestIsGitBackedDoltRemoteURL are by reading filepath.IsAbs. The probe reads the URL in the form dolt stores it: gitRemoteURLForLsRemote gives a scheme-less host/path.git the https:// dolt gives it, where git alone would read host:1234/repo.git as an scp address and probe a different endpoint. reset-data deletes the info ref dolt publishes under a DOLT_REMOTE_INFO_BRANCH override rather than the literal default, and none when an empty override disables the marker, since that name is then an ordinary branch; and the probes' and reset-data's git ls-remote and git push take the URL after --. The embedded cmd/bd shard manifest main now checks in lists the four TestEmbeddedGitDataRef* tests.

Why

Fixes #6261. Dolt takes the ref as a remote parameter (--ref on dolt remote add, dolt clone, DOLT_REMOTE, DOLT_CLONE); bd did not pass it through. Two uses: a git host that only accepts pushes under refs/heads/, and one repository holding one Dolt database per unit of work on refs such as refs/dolt/units/<key>.

Built on #6521 as merged (3efcd3571, with the maintainer fixup 0801749d6), rebased onto main at d652def9d. The command layer meets the fixup at three seams and changes nothing about them: bd dolt remote add --ref and bd bootstrap --ref validate the ref in the argument phase with the stricter command-level rules, so CreateRemoteWithRef's and BootstrapFromRemoteWithDBRef's add-time checks are a second gate that a well-formed ref passes; every ref the commands read comes from ListRemotes, which now goes through the one GitRefFromParams decoder, so reset-data and the re-add comparison see what the storage layer sees, and a SQL-visible remote record with an unreadable git_ref is an error before either acts. The one read that is not fail-closed is main's own: in the GH#2118 window, where a fresh sql-server lists no remotes, the re-add path falls back to the persisted repo_state.json through persistedRemoteInfosFor, which warns and assumes no remote on any read error, a malformed git_ref included, and the add then proceeds as it would for an absent remote; that fallback and its warning are main's and are unchanged here. The three follow-ups the adoption review on #6521 filed in the maintainers' ledger and quoted there (bd-2dozu, the untested rollback direction of the mirror probe; bd-xg5in, the proxied asymmetry argued from a code walk; bd-cbr9s, the repo_state.json flush and the pinned already exists text) are storage-layer and are not taken here.

Three rebases brought the branch from the pre-fixup #6521 head to 2704d8a95. The first moved 170 commits of main underneath and needed four hand merges: CHANGELOG.md (both entries kept); cmd/bd/config.go, where main extracted runConfigUnsetYamlOnly and had taught unset the nested spelling (#6574), which made the PR's own nested-clear hook for sync.remote-ref redundant, so the hook is gone and the PR no longer touches that file; cmd/bd/sync_git.go, where main's #nosec G702 note is kept over the ref argument; cmd/bd/bootstrap.go, where #6503's refuse action joins the Blocked derivation in detectBootstrapActionForRef and #6938's explicit BEADS_DIR branch sits beside the ref-following origin probe. The second moved 17 commits and needed one hand merge, the reset-data prompt line where #5795's reworded sentence met this PR's resetDataTarget call (both kept). The third moved 15 commits and needed two, both in CHANGELOG.md (entries ordered under ### Added and ### Fixed). The fourth, after the first review, moved 36 commits and needed one: #5760 made config.UnsetYamlConfig return whether the key was set, inside the block this PR had already replaced with clearOriginSyncConfig, whose body now takes the two values. The fifth moved 107 more, to 5e7367086, and needed one hand merge, in CHANGELOG.md again (the ### Fixed entry's neighbors). The sixth moved 4 more, to d652def9d, Bazel and CI changes with no file in common with this branch. git range-diff across each rebase showed nothing else moved; the audit comments on this pull request list what the adversarial rounds between the pushes changed.

The third commit is a fix on main's behavior that the rebase surfaced: an adversarial round on the rebase delta reproduced the reset-data ordering hazard against #5795 (ref deleted, then the commit failed, remote left without its data ref). It is separable; if the maintainers would rather take it alone, the first two commits stand without it.

Verification

Both ref shapes, refs/heads/issue-data and refs/dolt/units/team-12542, run through the end-to-end tests; the pure tests use them where a ref is compared or validated.

./scripts/test.sh -run 'TestGuardedSyncRemoteRef|TestGuardInitDoltRemoteRef|TestOriginPathsWithoutPromptRunGuard|TestGitRemoteURLForLsRemote|TestDoltInfoRefFollowsOverride|TestResetDataOrder|TestResolveSyncRemoteRef|TestValidateGitDataRef|TestDoltRemoteInfoRefPinsAgree|TestIsGitBackedDoltRemoteURL|TestRefForAdoptedRemote|TestDataRefForRemoteURL|TestGitOriginDoltURL|TestSyncRemoteRefIgnoredHint|TestEnsureDoltRemote|TestGitRemoteHasDoltDataRefAtStatus|TestDetectBootstrapAction|TestIsGitCodeRepoURL|TestPersistBootstrapRef|TestFinalizeSyncedBootstrap|TestCloneFromRemote|TestResolveRemoteCloneMode|TestConfigSideEffect|TestCheckConfig|TestCurrentOriginRemote|TestReplaceOriginRemote|TestApply|TestProbeGitDataRefTarget|TestGuardGitDataRefTarget|TestEnsureDoltRemoteGuarded|TestBootstrapPlan|TestPrintBootstrapPlan|TestResetData|TestProxiedInitDoltRemoteURL|TestPersistBootstrapRemoteAndRef|TestExplicitRefNote|TestDryRunRemoteApply|TestRedactRemoteURL' ./cmd/bd/   # ok (the probe and guard tests build a bare git repository and need git on PATH; TestDetectBootstrapAction covers main's probe-routing cases with the probe stubbed and the ref-following cases with the ref each probe asked for recorded; TestResetDataTargetNamesTheRef covers the reset-data naming for both ref shapes, the default, and the three non-git kinds)
./scripts/test.sh ./internal/config/   # ok (the ref validator, the info-branch override, the yaml value check)
BEADS_TEST_EMBEDDED_DOLT=1 ./scripts/test.sh -run 'TestEmbeddedGitDataRef' ./cmd/bd/   # ok, 4 tests, 80s: bd init with the committed key on the origin's default branch initializes, says it is not wiring origin there, configures no remote, and leaves the branch tip as it was; a row left pending (auto-commit off) before reset-data reaches the rebuilt ref and is read back from the workspace bootstrapped from it; with the dolt CLI on PATH, a foreign-key violation planted with checks off and recorded by `dolt constraints verify` makes reset-data fail on the commit with the ref unchanged (the test-embedded-cmd lanes have no dolt, so there that case logs and skips; TestResetDataOrder is the lane-independent pin: it drives the command with a fake store against a real bare git remote, embedded commits before it pushes and a failing commit leaves the planted ref and never pushes, server mode checks twice around the prompt and refuses a working set dirtied between the checks); remote add --ref then push lands on the ref and not on refs/dolt/data, reset-data without --yes and without a terminal refuses naming the ref and leaves it in place, reset-data --yes rebuilds that ref; bootstrap from sync.remote-ref and from --ref clones and records the ref; bd init in a git clone whose committed config carries the key clones from the ref and lists origin on it; bootstrap --ref against a .git-suffixed git+file:// remote with no data yet plans a fresh database, and running it wires origin on the ref and persists the key, also when sync.remote comes from BD_SYNC_REMOTE and no config.yaml existed; a key beside a file:// sync.remote is dropped from the plan and an explicit --ref there is refused; the origin probe sees data on refs/dolt/units/<key> only with the ref; a ref change without a terminal is refused without --yes and applied with it; --ref refs/dolt/data clears the key, so does bd config unset in the nested spelling; remove origin keeps the key and the next add lands on it; --ref refs/heads/main is refused as the default branch and an existing branch needs --yes; --ref on file:// is refused by bd; a malformed --ref is refused with no workspace at all; push-time adoption follows the key; bd config set applies the ref rules
go vet -tags gms_pure_go ./cmd/bd/... ./internal/config/... ./internal/storage/...   # ok; also CGO_ENABLED=0 GOOS=windows for cmd/bd and internal/config, GOOS=js GOARCH=wasm for internal/config
CGO_ENABLED=0 go test -tags gms_pure_go -c -o /dev/null ./cmd/bd/   # ok (the no-cgo test-binary compile pr.yml runs; the new test file carries //go:build cgo like the helpers it uses)
git diff --check, ./scripts/check-doc-freshness.sh, go test -tags=gms_pure_go ./test/docsync   # all pass
golangci-lint run --config=.golangci.yml --build-tags=gms_pure_go ./cmd/bd/... ./internal/config/...   # 1 issue, main's: unparam on primeGitCmd in cmd/bd/prime.go (892a438f1), a file this branch does not touch; 0 on the branch's own files
python3 tools/bazel/go_srcs.py --check   # ok (managed filegroups only); the seven new Go files are listed in cmd/bd/BUILD.bazel and internal/config/BUILD.bazel in gazelle's order, by hand (no Bazel on this machine), with no new external dependency; the bazel workflow's gazelle sync check has not run on this head yet and is the one to watch
make test   # 117 packages ok on this head, no failure, with the dolt CLI on PATH

Not exercised by any test here: the ref through cloneViaServer and cloneViaCLI in server mode; the CreateRemoteWithRef call itself inside configureProxiedInitDoltRemote's transaction in proxied-server mode; applyRemote's dry run against a real store (its decision is dryRunRemoteApply, pinned by TestDryRunRemoteApply against a bare repository); and bd init's prompt under a terminal and its consent wiring (the guard takes the resolved interactivity and the confirmed-divergence flag as booleans, pinned by TestGuardInitDoltRemoteRef; no test drives bd init --discard-remote --destroy-token with a key, and the end-to-end case runs with --non-interactive). Each is a direct parameter pass into the primitives of #6521, checked by reading; everything up to that call, the routed URL, the configured ref, and the guard, runs in createProxiedOriginRemote, which TestOriginPathsWithoutPromptRunGuard drives against a bare repository beside bd config apply's addOriginRemote (default branch and existing ref refused with a terminal attached and nothing created, free ref created on that ref), and TestProxiedInitDoltRemoteURL pins the routing. The end-to-end tests are embedded-mode only.

@sdghbtn

sdghbtn commented Sep 13, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review before this leaves draft: three rounds under each of two models (six in all), each round with a fresh context in a throwaway worktree of the branch, the diff against the #6521 commit it is stacked on, the body, and the prior rounds' dispositions.

Findings: 45 in total (2 blocker, 22 major, the rest minor or question). 44 accepted and fixed on the branch (four of them by withdrawing an internal/config unset change that #5760 owns), 1 rejected as framed (why the bootstrap guard yields to a stated ref rather than to a git+ scheme; the body records the reasoning). The main changes the rounds drove: git check-ref-format rules for --ref, with dolt's remote info branch (under its DOLT_REMOTE_INFO_BRANCH override) refused as a data ref; a git ls-remote --symref guard that refuses the repository's default branch and asks before an existing branch or tag, since dolt's first push replaces the ref's tip with Dolt storage; one reader for sync.remote-ref across the flat and nested spellings; a git-backed predicate that follows dolt's NormalizeGitRemoteUrl; the ref dropped with a warning, or an explicit --ref refused with a non-zero exit, beside a sync.remote that cannot carry one; the key surviving bd dolt remote remove origin (a round-1 change reversed in round 3, once remove && add was shown to move the data destination with no prompt); bd config apply and push-time adoption carrying the ref; the end-to-end tests renamed into the CI shard pattern and extended with bd init, reset-data, and the refusals; and the body corrected where it overstated coverage (the server and proxied-server clone paths are read here, not run).

@sdghbtn

sdghbtn commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review of this push, on the delta since 091d64e1e: the rebase across 48 commits of main onto the new #6521 head, then three rounds under each of two models, six in all, each round a fresh context in a throwaway worktree of the branch with the diff against the #6521 commit it is stacked on, the body, and the earlier rounds' dispositions.

Findings: 32 in total, 0 blocker, 3 major, the rest minor or question. 30 accepted and fixed on the branch; 1 reported as pre-existing on main and not touched here, executeRestoreAction runs DOLT_BACKUP('restore') without --force after creating the database, so the restore action fails in embedded mode; 1 named in the body as a follow-up, the /./ dolt keeps in a stored scp-style URL is not folded by bd's URL comparison. What the rounds drove: since #6037 replaced the old code-repository guard with a probe, the probe follows the ref everywhere, and the ref rides only plans that clone from or wire a remote, with the blocked plan carrying the ref its git ls-remote hint needs; the text plan names the remote and the ref a local plan wires before the user confirms; a branch or tag the probe finds is reported as found, not as Dolt data, on both probe legs; the local actions record the key and a credential-free sync.remote before origin is wired, and a persistence failure names the bd dolt remote add that finishes the job; an explicit --ref refs/dolt/data clears a configured key on every plan and the note says so; proxied init decides the ref against the routed URL and stores an scp-style URL as written, since bd's rewrite to git+ssh:// drops its relative path; the stale-key hint judges the routed form; the tests for each of those fail when their fix is reverted.

Verification on this head: the pure cmd/bd and internal/config suites named in the body, the four TestEmbeddedGitDataRef* end-to-end tests on the embedded engine, go vet, golangci-lint 0 issues on the changed packages, make test with one unrelated failure in scripts/repro-dolt-prod-timeouts that passes alone and that neither commit touches.

@sdghbtn

sdghbtn commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review of this push, on the delta since 6dc0dd427: the rebase across 170 commits of main onto the merged #6521 (with its maintainer fixup), the second commit on reset-data, and then three rounds under each of two models, six in all, each round a fresh context in a throwaway worktree of the branch with the full diff against main, the delta since the previous round, the body, and the earlier rounds' dispositions.

Findings: 22 distinct across the six rounds (the two models overlapped on five), 1 blocker, 6 major, the rest minor or question; 21 accepted and fixed on the branch, 1 rejected with the reason below. What the rounds drove: the seven new Go files are listed in the Bazel manifests main now gates on; the PR's nested-unset workaround for sync.remote-ref is gone because #6574 taught bd config unset the nested spelling, and the PR no longer touches config.go; the empty-value clear keeps its design with its real reason stated and enforced: the workspace-scoped readers resolve the key from the workspace's config.yaml first in either spelling, so a committed empty value pins the default even when the user-global config.yaml names a ref in the flat spelling that would otherwise shadow it through the merged reader; bd config set sync.remote-ref "" says it pins the default and bd config unset says the next configured value is read; a refuse plan carries no orphaned ref; reset-data --json reports the ref in canonical form with the deleted refs by name, pinned end to end; the PR's fixtures write config.yaml through bd's own writer, with the two hand-written shapes that are the point of their tests marked as such. Rejected: special-casing this key for a committed file whose whole root mapping is indented, which generic unset does not reach on main either and the open #5760 owns.

Verification on this head: the pure cmd/bd and internal/config suites named in the body, the four TestEmbeddedGitDataRef* end-to-end tests on the embedded engine, go vet, golangci-lint 0 issues on the changed packages, python3 tools/bazel/go_srcs.py --check, and make test (105 packages ok, none failing).

@sdghbtn
sdghbtn force-pushed the feat/dolt-remote-git-ref-cmd branch from cccbd8b to 810c3af Compare October 1, 2026 06:28
@sdghbtn

sdghbtn commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Audit of the push from cccbd8b73 to 810c3af19, for anyone reading the force-push.

Base. main moved twice under the branch since the last push: 17 commits to d0743f89d, then 15 to 2704d8a95. Both rebases are content-neutral except for three hand-merged hunks: the reset-data prompt line, where #5795's reworded sentence ("including any pending changes") met this PR's resetDataTarget call (both kept, and the sentence is now chosen by store mode, see below), and two CHANGELOG.md hunks (entries ordered under ### Added and ### Fixed). git range-diff across each rebase shows nothing else moved in the first two commits.

New third commit, fix(dolt): reset-data commits pending changes before it touches the remote. An adversarial round on the rebase delta reproduced an ordering hazard that #5795 created on main: every embedded push now commits pending working-set changes first, and reset-data deletes the remote's data ref (or clears the file store) before it pushes, so a commit that fails at that point (a constraint violation in the working set) left the remote without its data ref. This holds for refs/dolt/data on main today, not only for a custom ref, so the commit is separable; the first two commits stand without it if the maintainers would rather take it alone.

What it does: for a store without storage.PendingChangeDetector (embedded), CommitPending runs after the confirmation and before anything on the remote is removed, and a failure returns with the remote untouched and a hint naming bd dolt commit. For a store with the detector (server mode, the only implementer is *dolt.DoltStore), the existing dirty-working-set refusal runs before the prompt and again after it, because another session can write while the prompt waits; server mode never commits on the user's behalf (its push does not commit either). The prompt's pending-changes sentence follows the store mode; the help text names both behaviors instead of promising a refusal the embedded store never made. CHANGELOG gains a ### Fixed entry.

Rounds on the third commit, five in all across two models, and what they changed:

  • A first version committed unconditionally after the prompt; a reviewer showed at rung 4 (disposable Dolt server, a row inserted from a second session while the prompt waited) that it committed and published that session's work. Fixed as described above.
  • The only assertion pinning the ordering ran behind an exec.LookPath("dolt") gate, and the test-embedded-cmd lanes run cmd/bd with BEADS_TEST_EMBEDDED_DOLT=1 and no dolt, so CI never ran it (a reviewer confirmed at rung 4 that the end-to-end test passed with the fix reverted and dolt off PATH). Planting the violation through embeddeddolt.OpenSQL was tried and does not record a violation through the embedded driver on dolt 2.3.2 (four variants; dolt_constraint_violations stays empty while the CLI dolt constraints verify --all records it). So the end-to-end case stays CLI-gated as the real-engine check, and TestResetDataOrder is the lane-independent pin: it drives the command's RunE with a fake store against a real bare git remote; embedded commits before it pushes and a failing commit leaves the planted ref and never pushes; server mode checks twice and refuses a working set dirtied between the checks. With the CommitPending hunk removed in a scratch worktree, the two embedded cases fail and the server cases stand.
  • The stderr match in the end-to-end case also matched the bug's output (push after reset failed (...): commit pending before push); it now matches commit pending changes before reset.
  • The test file called helpers that live in //go:build cgo files, breaking the CGO_ENABLED=0 test-binary compile that pr.yml runs; the file carries the same constraint now.
  • Smaller: the hint on the embedded failure, the mode-dependent prompt sentence, the detector-to-mode coupling stated in the guard's comment, fix instead of feat, "never" instead of "no longer" in the message, the test comments and file header saying what each check pins (the pending-row read-back pins publication only).
  • Rejected: keying the branch on a new server-mode property instead of the detector (the first guard used the detector before this PR; the coupling is stated where it lives); a committed key in the --json result (bd dolt push is equally silent since fix(embeddeddolt): commit pending changes before push (#5433) #5795).

Verification on 810c3af19 is in the body's Verification block, including the no-cgo compile, the windows and wasm vets, make test (106 packages ok; cmd/bd failed once on TestGeneratedHookTimeoutProcessBoundary, a base test around a 1 s coreutils timeout under a loaded machine, and passed on a full rerun), and the one lint hit on this base (unparam on primeGitCmd in cmd/bd/prime.go, a file this branch does not touch).

@sdghbtn
sdghbtn marked this pull request as ready for review October 1, 2026 08:17
@sdghbtn
sdghbtn requested a review from a team as a code owner October 1, 2026 08:17
@sdghbtn

sdghbtn commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Out of draft. @bee-ghosttrack: this is rebased onto main over the merged #6521 with its maintainer fixup, the branch is three commits (the feature; reset-data naming the ref it replaces in its prompt, refusal, and JSON; and reset-data committing pending changes before it touches the remote), and the ref-blind reset-data you flagged on #6521 is closed here: it reads the ref from the remote record through the same decoder as everything else, deletes only that ref and the info ref, and names the ref before it acts. @julianknutsen: the third commit is a fix on main's behavior that the rebase onto #5795 surfaced (the delete-then-push order could leave the remote without its data ref when the push's new auto-commit failed) and is separable if you would rather take it alone; the three follow-ups from the adoption review are storage-layer and are not taken here, and the body's Why section says which and why.

@github-actions github-actions Bot added the status/needs-triage Awaiting initial PM triage label Oct 1, 2026

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 810c3af190 (merge-base 2704d8a954; main is 15 commits further at bcf4da3c65 and merge-tree is clean, the four overlapping files being additive on different keys). The ref-blind reset-data from #6521 is closed the way the out-of-draft note says, and the third commit's ordering holds under mutation. One thing to fix before this stands on its own, one small one, and notes.

Ran here (go1.27.1, dolt 2.3.5), all ok unless noted: go build ./...; go vet -tags gms_pure_go on cmd/bd and internal/config; CGO_ENABLED=0 go test -tags gms_pure_go -c ./cmd/bd/; python3 tools/bazel/go_srcs.py --check; gofmt on the 30 changed Go files; the body's 31-group -run on ./cmd/bd/ (148 PASS, 0 FAIL, 0 SKIP); go test ./internal/config/; BEADS_TEST_EMBEDDED_DOLT=1 -run TestEmbeddedGitDataRef on the real engine with the dolt CLI on PATH (4 PASS, 102s, no skip, so the planted-violation case ran). golangci-lint did not run here (the local v2.12.2 binary is built with go1.26 and panics on go1.27 sources); CI's Lint is the authority for that.

Mutations, each in a scratch worktree, reverted:

  1. CommitPending hunk removed: TestResetDataOrder embedded cases FAIL (calls = [push]; returned <nil>), server cases PASS. Exactly the claim.
  2. Commit moved after the remote delete: "embedded commit failure leaves the remote's ref in place" FAILS (refs/dolt/units/order changed … -> "").
  3. Post-prompt server recheck dropped: both server cases FAIL.
  4. lsRemoteDoltDataRefs handed "" instead of the recorded ref: TestEmbeddedGitDataRefRemoteAddPushesToRefAndBootstrapReadsIt FAILS on deleted_refs ([refs/heads/__dolt_remote_info__], want the data ref). Without BEADS_TEST_EMBEDDED_DOLT=1 that pin SKIPs; CI sets it in main.yml and pr-risk.yml, so it is run.

1. MAJOR, please fix — the default-branch / existing-ref guard covers two of the four paths that create origin on the configured ref

guardGitDataRefTarget is called from dolt.go:484 (push-time adoption) and dolt.go:1842 (remote add). Two more paths wire origin on sync.remote-ref with no probe in front of them:

  • config_apply.go:367-368: a fresh origin takes syncRemoteRefForURL(federationRemote) and calls AddRemoteWithRef directly. The comment above it says it "follows sync.remote-ref like every other path that creates one (init, bootstrap, push-time adoption)" — but init and bootstrap only wire a remote after their probe found the ref absent, and adoption runs the guard; apply has neither.
  • init_proxied_server.go:425: CreateRemoteWithRef(ctx, "origin", routed, syncRemoteRefForURL(routed)), same.

ValidateGitDataRef is format-only, so bd config set sync.remote-ref refs/heads/main is accepted. Commit that, then bd config apply on federation drift (or a proxied bd init) wires origin on the forge's default branch, and the next bd dolt push does not adopt (origin exists), so the guard never runs; by the help text's own words the push replaces the branch tip with Dolt storage. The guard's non-terminal branch already errors instead of prompting, so running it on these two paths fails closed without a UX change; if you would rather apply never prompt at all, pass a confirm that returns false. A test like TestEnsureDoltRemoteGuardedRunsGuardOnWritesOnly for each would pin it.

2. MINOR, please fix — the guard and the adoption warning print the raw URL

git_data_ref_target.go:67, 71, 77 and dolt.go:496 format url / remoteURL as given. The adoption path's URL is the git origin, which can carry https://x-access-token:<tok>@…. This PR's own redactRemoteURL (sync_remote.go:294, used 16 times in bootstrap.go and in init.go's ref branch) is the convention; these four sites and init.go:3386's no-ref branch miss it.

Notes, not asks

  • Windows drive letters. isGitBackedDoltRemoteURL (sync_remote.go:139-143) returns from the colon branch for any URL with a colon past position 0, so C:\repo\data.git is false (host C has no dot or @) and never reaches the local-path rule below. The comment says the drive letter must not be mistaken for a host, which is right, but the branch returns false there instead of falling through. If dolt's looksLikeLocalPath accepts that form, bd refuses --ref on a remote dolt treats as git-backed, and bootstrap drops a committed key with a warning. Worth one Windows-shaped case in TestIsGitBackedDoltRemoteURL.
  • gitDoltInfoRef (dolt_remote_reset_data.go:27) is the literal refs/heads/__dolt_remote_info__, while the PR adds the env-aware config.DoltRemoteInfoRef() and uses it for validation only. Under a DOLT_REMOTE_INFO_BRANCH override reset-data deletes the wrong marker ref; harmless in the normal case (dolt's marker push is best-effort), and the validator already refuses the override's branch as a data ref, so this is consistency, not safety.
  • URL-first argument injection into git ls-remote predates this PR and the new probe does not add a - refusal on the URL (the ref side is covered: refs/ prefix required, ValidateGitDataRefArg refuses a leading dash). The URL comes from the command line or .git/config, not committed config, so hardening only; a one-line refusal would match the ref side.
  • replaceOriginRemote (config_apply.go:273) keeps current.Ref while the add path takes the configured key, so a ref-only change in committed config is never "drifted". Defensible, and the two paths differ; the _ = on the restore-after-failed-add is pre-existing.
  • remote add origin without --ref resolves through the merged reader, so a user-global sync.remote-ref lands in the committed workspace config.yaml, and beside a non-git URL it refuses rather than drops with a warning. The body's "dropped with a warning" is bootstrap/init only; fine, just narrower than stated.
  • Proxied init now stores proxiedInitDoltRemoteURL(remoteURL) (a forge URL without .git becomes its git+ form), pinned by TestProxiedInitDoltRemoteURL. That is a routing change beyond the ref and belongs in the body.
  • The probe-then-add window is inherent and the probe is advisory; nothing to do.

Everything else I checked holds as described: argument-phase validation on both commands before storage opens, the sync.Once warning on a malformed committed value, the explicit-ref refusal and configured-ref drop on non-git URLs, no guard on a no-op re-add, reset-data reading r.Ref from ListRemotes through the shared decoder with a non-string git_ref erroring and persistedRemoteInfosFor never consulted, parseDoltDataRefs keeping exact matches only, the reader order (env, local, workspace in both spellings, merged) with workspace winning over user-global, the key surviving remote remove, the ref cleared on refuse plans, and //go:build cgo on the new helper-dependent test files.

Requesting changes on 1; 2 while you are there. Happy to re-read the delta.

@gascityinc-sarah gascityinc-sarah Bot added kind/feature New user-facing capability priority/p2 Medium: real problem with lower urgency labels Oct 1, 2026
@gascityinc-sarah

Copy link
Copy Markdown

Triage: kind/feature · priority/p2 — Adds --ref and sync.remote-ref so git-backed Dolt remotes work on hosts allowing only refs/heads/*. Command layer for merged #6521, fixes #6261: a known pain point with a workaround, so p2.
Auto-triaged by @gascityinc-sarah. Reply or relabel if this is off.

@sdghbtn

sdghbtn commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Audit of the push from 810c3af19 to 3ce1096f6, and the answer to the review of 2026-10-01, for anyone reading the force-push.

Base. main moved 147 commits under the branch since the last push, in three steps: 36 to e1ceedf09 (one hand merge: #5760 made config.UnsetYamlConfig return whether the key was set, inside the block this PR had replaced with clearOriginSyncConfig, whose body now takes the two values), 107 to 5e7367086 (one hand merge, in CHANGELOG.md; and the first commit now carries .github/scripts/embedded-cmd-test-shards.txt, which main started checking, listing the four TestEmbeddedGitDataRef* tests), and 4 to d652def9d (Bazel and CI changes with no file in common with the branch). git range-diff across each shows nothing else moved in the first three commits.

New fourth commit, fix(dolt): guard every path that wires origin on the configured ref; redact URLs, the answer to the review. Point by point:

  • 1 (major, the guard covers two of the four paths). Taken, and it was wider than the two named. bd config apply on a fresh origin and on a URL change, and proxied bd init, now run the guard before they wire origin, through a refusal shape for paths that cannot ask (refuseGitDataRefTarget): the default branch, an existing ref, and a ref the probe cannot read are all refusals, each naming the remedy of its own path (a bd dolt remote add origin ... --ref ... --yes where the workspace owns its store, with --allow-git-origin when the URL is the git origin; a bd sql call of DOLT_REMOTE over a proxied server, where remote add is refused; for a move by bd config apply, whose ref comes from the remote record rather than the key, a re-add of origin with --yes or with another --ref), shell-quoted and credential-free. The dry run of bd config apply runs the same read-only probe and reports the refusal the live run would make as an error result instead of promising an add. Two more paths turned up under review. Direct bd init wired origin on the ref whenever it did not bootstrap from the remote, and the premise that its probe stood in front of that was false: for a branch holding source the probe reports found, the clone fails with "no Dolt data", and init fell through to a fresh database with origin on that branch (reproduced: bd init exit 0, the next push rewrote the branch tip). It now asks with a terminal in interactive mode (the resolved value, so BD_NON_INTERACTIVE and CI count), refuses otherwise, takes an existing ref without asking when the user authorized the divergence in the destroy-token flow (the default branch stays refused), and in every case leaves origin unconfigured, names the bd dolt remote add origin ... --ref ... that finishes the job, and goes on, as it does when the add itself fails. (A round reproduced why the remedy has to be the add and not a change to the key: init does not run twice, and the next bd dolt push adopts the git origin rather than sync.remote, so with the two pointing at different repositories the database landed on the code repository and sync.remote was rewritten to it. Adoption preferring the git origin over a configured sync.remote is main's behavior; the body names it as a follow-up.) And guardGitDataRefTarget itself admitted the ref when the probe failed, before any terminal or --yes check, so a setup script's remote add without a terminal and a CI bd dolt push --yes could wire the default branch of a repository they could not read; a failed probe is now a refusal without a terminal, --yes or not, and warns and goes on only with one. Pinned by TestGuardGitDataRefTarget (the failed probe), TestGuardedSyncRemoteRef, TestGuardInitDoltRemoteRef, TestOriginPathsWithoutPromptRunGuard (apply's addOriginRemote and createProxiedOriginRemote against a bare repository), TestReplaceOriginRemote, TestDryRunRemoteApply, and an end-to-end case where bd init with the key on the origin's default branch initializes, says why origin is not wired, and leaves the branch tip as it was.
  • 2 (minor, raw URLs). Taken at the four sites and init.go's no-ref line, and two more leaks the rounds found: the wrapped probe error carried the raw URL inside the redacted warning, and redactRemoteURL left a scheme-less user:token@host/repo.git intact, a form dolt reads as https. The probe's own error now goes through gitLsRemoteProbeError (git's first stderr line, credential-scrubbed, as the bootstrap probe reports it), and redactRemoteURL drops the userinfo of a scheme-less host/path that is neither scp-like nor a local path and spells the result https://, because the userinfo was what kept a dotted host before the first colon (host.example:443/repo.git) from reading as an scp address, for dolt and git alike, and bootstrap persists the redacted URL into sync.remote (a round reproduced the transport flip through BD_SYNC_REMOTE); TestRedactRemoteURL carries the cases. Two neighbors of those lines in the same flows, the adoption announcement under --yes and init's probe-failure note, printed the raw URL and go through it now. The url fields of remote list --json and reset-data --json, and the other messages main already printed, are as they were; the body's sentence names the sites.
  • Windows drive letters. Taken. isGitBackedDoltRemoteURL falls through from a colon that makes no scp host to dolt's path rules, so C:\repo\data.git is git-backed on Windows (and /srv/le:dgers.git everywhere); TestIsGitBackedDoltRemoteURL carries the cases by GOOS. The Windows lanes build and smoke-test rather than run this package, so those expectations are by reading filepath.IsAbs. It holds for the URL as given to remote add; a git origin still goes through doltremote.Normalize first, which spells a drive-letter path git+C:\..., a pre-existing shape neither bd nor dolt reads as git-backed, and the body says so.
  • gitDoltInfoRef. Taken. reset-data deletes the info ref dolt publishes under the DOLT_REMOTE_INFO_BRANCH override, and none when an empty override disables the marker, since the default name is then an ordinary branch that may hold data (a round showed the earlier shape deleting refs/heads/__dolt_remote_info__ beside refs/dolt/units/a with the marker disabled). The local constant is gone; TestDoltInfoRefFollowsOverride and TestResetDataRefNamesMatchDolt (config's pin against dolt's) cover it.
  • URL-first argument injection. Taken: the probes' and reset-data's git ls-remote and git push take the URL after --.
  • replaceOriginRemote keeps current.Ref. Kept, as you called it, and it now runs the guard against the new repository before the remove, since the ref was admitted against the old one; its remedy names the re-add, because changing the key would not change what that path reads. The _ = on the restore is as it was.
  • remote add origin without --ref refuses rather than drops. As you read it; the body's first bullet already scoped the drop-with-a-warning to bootstrap, init, and apply, and names the add as the one path that refuses.
  • Proxied init's routing change. The body's bd init bullet names it as a change to what proxied init stores, not only to the ref.
  • The probe-then-add window. Agreed; nothing done.

One more thing the rounds found that the review did not name: gitRemoteURLForLsRemote gives a scheme-less host/path.git the https:// dolt gives it before the probe, because git alone reads host:1234/repo.git as an scp address and probed a different endpoint than the one dolt would push to (reproduced with a URL rewrite: the probe failed, the add went through without --yes, the push replaced the other repository's main).

Rounds, three under each of two models on the delta, each a fresh context in a throwaway worktree with the full diff against main, the delta since the previous round, the body, your review, and the earlier rounds' dispositions. Round one (on the first draft of the fourth commit): 9 and 5 findings, 2 blockers (direct bd init above; the scp-versus-https probe endpoint), 3 distinct majors (the wrapped probe error leaking the URL; a --yes remedy named on paths that have no --yes; the info ref deleted beside another database's data with the marker disabled); all taken. Round two: 9 and 6, 1 blocker (the fail-open probe on the unattended paths above), 2 distinct majors (the raw flag instead of the resolved non-interactive value in init; the scheme-less https leak); all taken, and one question answered by design: a divergence the user authorized in init counts as consent to take an existing ref, never the default branch. Round three: 9 and 2, no blocker, 3 majors (consent plus a pty under --non-interactive reaching the fail-open probe branch; the redaction's transport flip; init's remedy without an add step, above); 10 taken and 1 already covered by another's fix, the rest minor (the attended refusal now carries the remedy; the remedies are built on a refusal only, since building one reads the git origin; -- on the status probe too; the body's -run line and its qualifications).

Verification on 3ce1096f6 is in the body's Verification block: gofmt, build, vet (darwin, windows under CGO_ENABLED=0, wasm for internal/config), the no-cgo test-binary compile, the body's -run set, internal/config, the four end-to-end tests on dolt 2.3.2, golangci-lint on the branch's files, go_srcs.py --check, the shard manifest check, git diff --check, doc freshness, docsync, and make test (117 packages ok, no failure, with the dolt CLI on PATH). The Bazel workflow's own gazelle and nogo checks have not run on this head; they are the ones to watch.

@bee-ghosttrack: the delta since 810c3af19 is above; a re-read when you have time would be appreciated.

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-read at 3ce1096f68 (merge-base d652def9db; main is 1 commit further at 8d7d7fef16, the RBE worker-env pin, no file in common). The two items from 2026-10-01 are closed, the four paths are behind one guard, and the guard's own fail-open gap is shut. Approving.

The rebase. git range-diff from 810c3af190: commit 2 is patch-identical; commit 1 differs only by the four embedded-cmd-test-shards.txt rows and the two-value config.UnsetYamlConfig in bootstrap.go; commit 3 only by its CHANGELOG line. Exactly the three hand merges the push note names.

Ran here (go1.27.1, dolt 2.3.5), all ok: gofmt on the branch's Go files; go build ./...; go vet -tags gms_pure_go on cmd/bd and internal/config, and again GOOS=windows CGO_ENABLED=0; CGO_ENABLED=0 go test -tags gms_pure_go -c ./cmd/bd/; python3 tools/bazel/go_srcs.py --check; git diff --check; the body's -run set plus TestGuardGitDataRefTarget|TestProbeGitDataRefTarget|TestReplaceOriginRemote|TestDryRunRemoteApply|TestRedactRemoteURL (24 PASS, 0 SKIP); go test ./internal/config/; BEADS_TEST_EMBEDDED_DOLT=1 -run TestEmbeddedGitDataRef on the real engine (4 PASS, 190s, no skip; the new bd init refuses the key on the origin's default branch case ran under both ref shapes); scripts/ci/gen_embedded_cmd_shard_manifest.py --check 50 passes on this head and on main alike; the 31 changed Go files are each listed in their BUILD.bazel. golangci-lint did not run here (same toolchain mismatch as last time); the Bazel lanes on this head are the authority for nogo and gazelle, and at the time of writing Bazel / execution mode, dolt-server lane and integration are green with the six Dolt tiers still running on the fork pool.

Mutations, each in a scratch worktree on this head, reverted, each caught by the test the body names for it:

  1. addOriginRemote takes syncRemoteRefForURL without the guard → TestOriginPathsWithoutPromptRunGuard fails on the refs/heads/main add.
  2. guardGitDataRefTarget admits a failed probe without a terminal (the old branch) → TestGuardGitDataRefTarget fails at the yes=false refusal.
  3. redactRemoteURL returns a scheme-less URL raw → TestRedactRemoteURL fails on both token cases.
  4. guardInitDoltRemoteRef passes false for the consent on the unattended path → TestGuardInitDoltRemoteRef fails at "a consented divergence takes the existing ref"; passing true fails at "an existing ref must not be wired without a prompt". Both arms pinned.
  5. The colon branch of isGitBackedDoltRemoteURL returns false past the scp check → TestIsGitBackedDoltRemoteURL fails on /srv/le:dgers.git, ./le:dgers.git, C:/repo/data.git.
  6. replaceOriginRef skips the guard → TestReplaceOriginRemote's new case and TestDryRunRemoteApply's "move onto the default branch" both fail.
  7. createProxiedOriginRemote bypasses the guard → TestOriginPathsWithoutPromptRunGuard fails on the proxied refs/heads/main create.
  8. gitRemoteURLForLsRemote leaves a scheme-less host/path alone → TestGitRemoteURLForLsRemote fails, and TestProbeGitDataRefTarget shows the mechanism: ssh: Could not resolve hostname reviewhost, git reading reviewhost:1234/code.git as scp.
  9. doltInfoRef returns the old literal → TestDoltInfoRefFollowsOverride fails on the override and on the disabled marker.
  10. The bd init call site wires origin unconditionally (the guard call replaced by true) → the embedded bd init refuses the key on the origin's default branch case fails under both ref shapes: no refusal on stderr and origin configured on refs/heads/main. So the end-to-end pin reaches the call site, not only the helper.
    I also read dolt's NormalizeGitRemoteUrl, isScpLikeGitRemote and looksLikeLocalPath in the pinned module: isScpLikeForDolt and looksLikeLocalPathForDolt are faithful mirrors, and the colon branch's url.Parse("git+https://"+url) is the same fallthrough dolt takes, so host:org/repo.git (non-numeric port) is "not git-backed" here where dolt errors, and C:/repo/data.git is git+https on every non-Windows platform for both.

Point by point against the review: 1 (the two unguarded paths) is wider than I asked and closed on all four, with the dry run reporting the refusal it would make; the failed-probe fail-open was a real third gap and is the right call. 2 (raw URLs) covered at the four sites, init.go's no-ref line, the wrapped probe error, and the scheme-less userinfo form; spelling the result https:// is correct, since the userinfo was what kept host.example:443 from reading as scp. Windows drive letters, gitDoltInfoRef, -- on the URL, replaceOriginRemote keeping current.Ref (and now guarding against the new repository): all as described.

Two notes, not asks. (a) bd config apply on a URL change, and on a fresh origin, now returns an error result where the branch exists on the target or the probe fails, with the re-add as the way out; the previous behaviour wired origin silently, so this is the intended side of the trade, but it is a visible change for federation flows that moved a repository and already pushed the ref there, and the CHANGELOG line covers it. (b) The remedies call doltRemoteMatchesGitOrigin, which reads the git origin; they are built on a refusal only, as the body says, so the probe stays the only network read on the happy path.

@sdghbtn
sdghbtn force-pushed the feat/dolt-remote-git-ref-cmd branch from 3ce1096 to 930b655 Compare October 7, 2026 20:42
@sdghbtn

sdghbtn commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main at 77ab98079 (3ce1096f6 -> 930b655d8, force-pushed with lease). The branch now carries five commits; the four reviewed ones are unchanged in content.

Before the rebase I read the base's CI on 3ce1096f6. Three Bazel lanes were red. Two were the runner, not the branch: the pure-Go lane died on an UNAVAILABLE: HTTP status code 502 from remote execution while running nogo on a third-party dependency, and the proxied-server tier's bd_managed_local_test failed in 0.1s with unshare: unshare failed: No space left on device before any test ran (its test.xml lists zero tests). The third was ours. In the cmd/bd Dolt-server tier, shard 5, TestCLI_Import failed because its in-process bd init found a stale git context (IsWorktree() true, common dir under a TempDir that main's TestDetectBootstrapAction_WorktreeSynthesizedDirPrefersSyncOverDefaultSharedDB had already removed), so the workspace gate's parent did not exist and init refused. The cause was in this branch: detectBootstrapAction(beadsDir, cfg), the wrapper the planning tests call, resolved sync.remote-ref with resolveSyncRemoteRef(), which rediscovers the workspace from the process cwd through beads.FindBeadsDir() and primes the cached git context on the way. main's wrapper never touched that cache; rules_go shards by declaration index, so the tests this branch added moved TestCLI_Import into the slot right after the worktree test.

The fifth commit, 930b655d8, has the wrapper resolve the ref for the directory it is handed (resolveSyncRemoteRefIn(beadsDir): env, local layer, workspace file, then the merged configuration). The command path is unchanged; it already resolved the ref once and called detectBootstrapActionForRef. I proved it with a probe test, not committed, that chdirs into a fresh temp dir after the worktree test and asserts IsWorktree() is false: it fails on each of the four earlier commits and passes with the fifth and on main at both d652def9d and 77ab98079. The agent's review round on the fix found nothing major; its one accepted point was that the comment overstated the last fallback, which now reads as the process's merged configuration.

Rebase conflicts: one, .github/scripts/embedded-cmd-test-shards.txt, where main and this branch each added a shard-15 line; both kept in order. CHANGELOG.md and cmd/bd/BUILD.bazel auto-merged. Range-diff against the pre-rebase branch: = for commits 2, 3, 4; commit 1 differs only by that manifest line; commit 5 is new.

Gates on 930b655d8, all green: make test-go (117 packages ok, 0 failures, with the dolt CLI on PATH), go test ./scripts/ (the shard-manifest pins among them), golangci-lint 0 issues, go vet and gofmt, tools/bazel/go_srcs.py --check, the body's -run set, internal/config, the four TestEmbeddedGitDataRef* end-to-end tests with BEADS_TEST_EMBEDDED_DOLT=1, the CGO_ENABLED=0 test-binary compile, git diff --check, doc freshness, docsync. The base's CI on the new head is running as I write this.

The body still names d652def9d as the base and four commits; I left it alone so the review history stays readable, and this comment is the record of the delta.

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-read at 930b655d8 after the rebase; the approval at 3ce1096f6 carries over to this head.

The four reviewed commits. git range-diff d652def9d..3ce1096f6 77ab98079..9ca0bfd8d: commits 2, 3 and 4 are =; commit 1 differs only in .github/scripts/embedded-cmd-test-shards.txt (the shard-15 line main added beside yours, both kept), as your comment says. gen_embedded_cmd_shard_manifest.py --check 50 exits 0 at 930b655d8, exact-once coverage; a main-side control exits 0 too.

The fifth commit is one line in cmd/bd/bootstrap.go: the wrapper resolves through resolveSyncRemoteRefIn(beadsDir), which reads the env var, then config.WorkspaceYamlValueStrictWithLocal (plain os.ReadFile of the two YAML files under beadsDir), then the merged configuration. Nothing on that path reaches beads.FindBeadsDir or the git context cache. main's wrapper never resolved a ref, so the regression was this branch's alone, as you read it.

Both-SHA probe of the claim, since your probe test was not committed: a zz_probe_test.go declared after the worktree test, chdir into a fresh t.TempDir(), assert git.IsWorktree() is false; run as go test ./cmd/bd -count=1 -run 'TestDetectBootstrapAction_WorktreeSynthesizedDirPrefersSyncOverDefaultSharedDB$|TestZZProbeGitContextClean$' under GOTOOLCHAIN=go1.26.7.

  • 9ca0bfd8d (the four reviewed commits): worktree test PASS, probe FAIL — IsWorktree() true from fresh dir, the context primed by the previous test.
  • 930b655d8: both PASS.

Merge gate. At 20:55Z the Bazel tiers on 930b655d8 were still in progress (cmd/bd Dolt-server, embedded Dolt, proxied-server, server-Dolt storage, pure-Go and js/wasm, integration, test); dolt-server lane, execution mode and both Package Gates are green. The cmd/bd Dolt-server tier is the one that showed the shard-5 failure on the previous head, so that is the result that closes this out; I will read it when it lands.

@sdghbtn
sdghbtn force-pushed the feat/dolt-remote-git-ref-cmd branch from 930b655 to b1c84c1 Compare October 14, 2026 00:14
@sdghbtn

sdghbtn commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main at d4ca89369 (930b655d8 -> b1c84c1f5, force-pushed with lease). Still five commits.

One conflict, .github/scripts/embedded-cmd-test-shards.txt: main has since replaced the hand-edited 50-shard blocks with the generated 100-shard block (#7438, #7478), so I took main's file and ran the generator incrementally (scripts/ci/gen_embedded_cmd_shard_manifest.py 100 --weights=duration --write), which placed the four TestEmbeddedGitDataRef* tests on shards 61 to 64; --check passes and a second --write changes nothing. CHANGELOG.md, both BUILD.bazel files, bootstrap.go, init.go, context_cmd.go, and sync_remote_test.go auto-merged, and I read each seam against main's intent (the migration-consent line after the clone, the http backend branch in bd context).

One line changed beyond the rebase: targetTestGit in cmd/bd/git_data_ref_target_test.go now builds its command through the gitCommand helper #7358 introduced (maintenance.auto=false on argv), which every other cmd/bd test fixture had already moved to. The agent's review round on the rebase delta raised it; a second round found nothing. Range-diff against 930b655d8: = for commits 2 to 5; commit 1 differs by the manifest and that line.

Gates on b1c84c1f5, all green: make test-go (121 packages ok, 0 failures, with the dolt CLI on PATH), go test ./scripts/ (the shard-manifest checks among them), golangci-lint 0 issues, go vet and gofmt, tools/bazel/go_srcs.py --check, the body's -run set, internal/config, the four TestEmbeddedGitDataRef* end-to-end tests with BEADS_TEST_EMBEDDED_DOLT=1, the CGO_ENABLED=0 test-binary compile, git diff --check, doc freshness, docsync. The base's CI on the new head is running as I write this.

One observation from the review, left alone because changing it is not a rebase: persistBootstrapRef's early return in executeSyncAction now sits before main's new schema.SetLocalMigrateConsent(true), the same window main's own finalizeSyncedBootstrap return already occupies.

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase read at b1c84c1 (force-push over my approval at 930b655), on a probe worktree at a printed HEAD, 0 dirty.

  • Merge base moved 77ab980 → d4ca893 (current main). git range-diff two-range form: commits 2–5 are =; only commit 1 differs.
  • Net-patch interdiff (manifest and CHANGELOG context excluded) has exactly two content deltas: the four new TestEmbeddedGitDataRef* manifest rows moved from the retired 50-shard block into main's 100-shard block (shards 61–64), and git_data_ref_target_test.go now calls the package's existing gitCommand(args...) helper (cmd/bd/helpers_test.go:74) instead of a raw exec.Command("git", …). Everything else is hunk-offset drift.
  • gen_embedded_cmd_shard_manifest.py --check rc 0 at the live 100-shard count (heaviest shard 156.7s).
  • go vet ./cmd/bd ok at the new head (tests compile); the four new tests by name under BEADS_TEST_EMBEDDED_DOLT=1: all four PASS (97s), HEAD b1c84c1, 0 dirty.

The approval stands on this head.

@bee-ghosttrack

Copy link
Copy Markdown
Collaborator

Maintainer note (bee): mergeStateStatus is DIRTY again since main moved past your 00:14Z rebase (#7553, #7554, #7552 landed; main 70887e7). Local probe at b1c84c1: git merge-tree origin/main HEAD → CONFLICT (content) in .github/scripts/embedded-cmd-test-shards.txt only, the shard manifest's live block. My re-approval (5511607095) stands at b1c84c1; a rebased head gets a fresh interdiff read. When you rebase onto 70887e7, regenerate the manifest with the script rather than hand-merging the block.

…-ref

Dolt takes the git ref a git-backed remote keeps its data on as a remote
parameter (--ref on dolt remote add, dolt clone, DOLT_REMOTE, and
DOLT_CLONE); the storage layer passes it through since the previous
commit, and this commit exposes it. Two uses: a git host that only
accepts pushes under refs/heads/, and one repository holding one Dolt
database per unit of work on refs such as refs/dolt/units/<key>.
Fixes gastownhall#6261.

- bd dolt remote add --ref <ref>: a full ref only (refs/...), refused
  for a URL that is not git-backed before dolt sees it. For origin the
  ref is written to .beads/config.yaml as sync.remote-ref, read back
  when origin is re-added without --ref, and cleared by --ref
  refs/dolt/data or bd config unset.
- A ref change replaces the remote only after confirmation; without a
  terminal it is refused with both refs named unless --yes is given.
- bd bootstrap --ref overrides the key; the plan carries and prints the
  ref, all three clone modes pass it through, the probe of a git
  sync.remote or of the git origin looks at that ref, a fresh database
  created when the ref holds no data yet is wired to origin on it with
  the key persisted (config.yaml created first, and sync.remote written
  beside the key, when the remote came from the environment; both are
  written before origin is wired, and a failure names the remote add
  that finishes the job), and the ref is persisted beside sync.remote
  after a clone. An explicit default --ref clears the key on every plan.
- bd init reads the key for its clone and for the origin it configures,
  including proxied-server mode; the ref is decided against the routed
  (git+) form of the URL; the proxied path stores that form for a URL
  dolt would not read as git-backed as written and leaves the rest, the
  scp form among them, as written; the stale-key hint on bd config set
  sync.remote judges the routed form too.
- bd dolt remote list shows the ref; bd dolt remote reset-data rebuilds
  the configured ref and deletes only that ref and the info ref.
- Docs: both commands' help, the configuration reference, a section in
  sync-setup for both uses, and a CHANGELOG entry.

- cmd/bd/BUILD.bazel and internal/config/BUILD.bazel list the new files;
  no new dependency.

- The ref that applies to a workspace is resolved in the order the
  configuration layers merge (BD_SYNC_REMOTE_REF, config.local.yaml,
  the workspace config.yaml in either spelling, then the user-global
  file), reading the workspace files directly so a committed empty
  value pins the default over a user-global ref; bd context, bd config
  get, and push-time adoption report and use that same value.
reset-data already reads the ref of a git-backed remote from the remote
record and deletes only that ref and the info ref, but the confirmation
prompt, the refusal without a terminal, and the JSON result named the
URL alone. One repository can hold several databases on their own refs,
and there the URL does not say which database is about to be replaced.

- The prompt and the non-interactive refusal name the URL and the ref
  (resetDataTarget); a remote that is not git-backed has no ref and is
  named by its URL as before.
- The JSON result gains ref as bd dolt remote add --json reports it: the
  recorded ref in canonical form when it is not the default, no key
  otherwise. bd dolt remote list --json shows the same for every remote
  bd created, since bd canonicalizes before recording; a remote added
  through dolt itself with an explicit --ref refs/dolt/data lists
  verbatim. The refs deleted, the default among them, stay in
  deleted_refs.
- TestResetDataTargetNamesTheRef covers both ref shapes, the default in
  both spellings, and the three non-git kinds; the end-to-end test runs
  reset-data without --yes and without a terminal first and checks the
  refusal names the ref and leaves it in place, then decodes the --json
  result and checks ref and that deleted_refs holds the remote's ref and
  never refs/dolt/data.
…emote

Since gastownhall#5795 every Push variant of the embedded store commits pending
working-set changes before it pushes. reset-data deletes the remote's
data ref (or clears the file store) and then pushes, so a commit that
fails at that point, a constraint violation in the working set for one,
left the remote without its data ref and the user with the "re-run bd
dolt push --force" hint and a remote no clone can bootstrap from. This
holds for the default ref on main today, not only for a custom one.

For a store with a private working set (embedded), commit pending
changes after the confirmation and before anything on the remote is
removed: a failing commit now returns with the remote as it was and a
hint naming bd dolt commit. For a store that reports pending changes
(server mode, where the working set is shared), keep refusing while
anything is uncommitted, and check again after the confirmation:
another session can write while the prompt waits, and committing here
would publish that session's unfinished work. The prompt's sentence
about pending changes follows the store mode, and the help text names
both behaviors instead of promising a refusal the embedded store never
made (its detector has been server-only from the start).

TestResetDataOrder drives the command with a fake store and a real bare
git remote: embedded commits before it pushes and a failing commit
leaves the planted ref and never pushes; server mode checks twice
around the prompt, refuses a working set dirtied between the checks,
and never commits. The end-to-end test leaves a row in the working set
(auto-commit off) before the reset and reads it back from the workspace
bootstrapped from the ref; with the dolt CLI on PATH it also plants a
foreign-key violation (checks off, recorded by `dolt constraints
verify`, which is what makes DOLT_COMMIT refuse) and asserts the reset
fails on the commit with the remote's ref unchanged.
…redact URLs

guardGitDataRefTarget ran on bd dolt remote add and on push-time
adoption, but the other paths that create origin on sync.remote-ref had
no probe in front of them: bd config apply adding a fresh origin on
federation drift or moving it to a new URL, proxied bd init, and direct
bd init whenever it wires a fresh database to the remote instead of
bootstrapping from it (the origin probe reports a branch holding source
as found, and the clone of it is what reports no data). And the guard
itself admitted the ref when the probe failed, before any terminal or
yes check, so a setup script's remote add and a CI push with --yes could
wire the default branch of a repository they could not read.
ValidateGitDataRef is format-only, so a committed refs/heads/main
reached each of them, origin was wired on the forge's default branch,
and the next push did not adopt (origin existed), so the guard never ran
and the push would have replaced the branch tip with Dolt storage.

- guardGitDataRefTarget treats a failed probe as a refusal without a
  terminal, yes or not, since yes cannot vouch for a default branch and
  nobody reads the warning on an unattended run; with a terminal it
  warns and goes on as before.
- refuseGitDataRefTarget is the guard for a path that cannot ask: the
  default branch, an existing ref, and a ref the probe cannot read are
  all refusals, each naming the calling path's remedy (gitDataRefRemedy:
  how to take an existing ref on purpose, how to move to a free one).
  The remedies paste into a shell as one command, URL and ref quoted
  and the URL credential-free: bd dolt remote add origin ... --ref ...
  --yes (with --allow-git-origin when the URL is the git origin) where
  the workspace owns its store; bd sql with a DOLT_REMOTE call over a
  proxied server, where remote add is refused; a re-add of origin for a
  move by bd config apply, whose ref comes from the remote record rather
  than the key. bd config apply runs it on a fresh origin
  (addOriginRemote), on a URL change (replaceOriginRef, against the new
  repository, since the ref was admitted against the old one), and in
  the dry run (dryRunRemoteApply), which reports the refusal the live
  run would make as an error result instead of promising an add; proxied
  bd init runs it in createProxiedOriginRemote; direct bd init asks in
  interactive mode with a terminal (the resolved non-interactive value,
  so BD_NON_INTERACTIVE and CI count), refuses otherwise, takes an
  existing ref without asking when the user authorized the divergence
  (destroy token or typed confirmation; the default branch stays
  refused, and so does a ref the probe cannot read whenever nobody is
  watching, a pty under --non-interactive included), and in every case
  leaves origin unconfigured, names the bd dolt remote add origin that
  finishes the job, and goes on, as it does when the add itself fails:
  init does not run twice, and the next push adopts the git origin
  rather than sync.remote, the right remote only when the two are the
  same repository, so the remedy is the add and never the key alone.
  The remedies are built on a refusal only, since building one reads
  the git origin. Pinned by TestGuardGitDataRefTarget
  (the failed probe), TestGuardedSyncRemoteRef, TestGuardInitDoltRemoteRef,
  TestOriginPathsWithoutPromptRunGuard, TestReplaceOriginRemote,
  TestDryRunRemoteApply, and an end-to-end case where bd init with the
  key on the origin's default branch initializes, says why origin is not
  wired, and leaves the branch tip as it was.
- The guard's messages, the probe's own error (git's first stderr line,
  credential-scrubbed, through gitLsRemoteProbeError), the adoption
  warning and the adoption announcement under --yes, init's no-ref
  confirmation and its probe-failure note print the URL through
  redactRemoteURL, which now also drops the userinfo of a scheme-less
  user:token@host/repo.git, a form dolt reads as https: the adoption URL
  is the git origin, which CI configures with a token in its userinfo.
  The result is spelled https://, since the userinfo was what kept a
  dotted host before the first colon (host.example:443/repo.git) from
  reading as an scp address, for dolt and git alike, and bootstrap
  persists the redacted URL into sync.remote and the remedies paste it.
- isGitBackedDoltRemoteURL falls through from a colon that makes no scp
  host to the path rules, as dolt's NormalizeGitRemoteUrl does, so a
  Windows drive-letter or UNC path ending in .git is git-backed on
  Windows and /srv/le:dgers.git is on every platform; before, the colon
  branch returned false there and bd refused --ref on a remote dolt
  treats as git-backed. TestIsGitBackedDoltRemoteURL carries the cases
  by GOOS; the Windows lanes build and smoke-test rather than run this
  package, so the Windows expectations are by reading filepath.IsAbs.
- gitRemoteURLForLsRemote gives a scheme-less host/path ending in .git
  the https:// dolt gives it, so a probe reads the repository dolt will
  push to; git alone reads host:1234/repo.git as an scp address, and a
  probe of the wrong endpoint failed and, on the interactive path,
  warned and admitted the ref. A git remote name such as origin is left
  alone, as is everything git reads already.
- reset-data deletes the info ref dolt publishes under the
  DOLT_REMOTE_INFO_BRANCH override, not the literal default, and none
  when an empty override disables the marker, since the default name is
  then an ordinary branch that may hold data;
  TestDoltInfoRefFollowsOverride. The local pin of that name is gone;
  TestResetDataRefNamesMatchDolt checks config's against dolt's.
- The probes' and reset-data's git ls-remote and git push take the URL
  after --, so a URL can never read as an option, matching the leading
  dash refusal on the ref side.
- The CHANGELOG entry says where the guard refuses.
…e directory it plans

detectBootstrapAction(beadsDir, cfg), the wrapper every planning test
calls, resolved the ref with resolveSyncRemoteRef, which rediscovers the
workspace from the working directory through beads.FindBeadsDir. The
command path never used it (bd bootstrap resolves the ref once and calls
detectBootstrapActionForRef), so in production the two directories were
the same, but the discovery reads the process-wide git context, and a
test that plans from inside a worktree fixture left that context primed
with the fixture's common dir after the fixture was removed.

The cmd/bd Dolt-server tier showed it on this branch: rules_go assigns
tests to shards by declaration index, the new test functions moved
TestCLI_Import into the shard right after
TestDetectBootstrapAction_WorktreeSynthesizedDirPrefersSyncOverDefaultSharedDB,
and its in-process bd init read IsWorktree()=true with a common dir
under a cleaned-up TempDir, so the workspace gate's parent did not exist
and init refused. On main the same pair leaves no context behind; on
every commit of this branch it did.

- The wrapper resolves the ref with resolveSyncRemoteRefIn(beadsDir):
  the environment, the local layer, the workspace file, then the merged
  configuration, for the directory it was handed, the merged layer being
  the process's. Nothing on the path
  touches the git cache any more, and a caller that plans a directory
  other than the one discovery would find reads that directory's key.
- Verified with a probe test (not committed) that chdirs into a fresh
  temp dir after the worktree test and asserts IsWorktree() is false:
  fails on each of the four earlier commits, passes with this one and on
  main; the body's -run set passes.
@sdghbtn
sdghbtn force-pushed the feat/dolt-remote-git-ref-cmd branch from b1c84c1 to d3dd0bc Compare October 18, 2026 15:33
@sdghbtn

sdghbtn commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main at ce70b261c (b1c84c1f5 -> d3dd0bc43, force-pushed with lease). 70887e708 is in that range; main had moved on by the time I read it. Still five commits.

One conflict, .github/scripts/embedded-cmd-test-shards.txt, resolved the way you asked: I took main's file and ran scripts/ci/gen_embedded_cmd_shard_manifest.py 100 --weights=duration --write, which placed the four TestEmbeddedGitDataRef* tests on shards 56 to 59 this time; --check passes and a second --write changes nothing. The twelve other files both sides touched auto-merged (CHANGELOG.md, cmd/bd/BUILD.bazel, bootstrap.go, init.go, the three init_safety* and init_proxied_server.go, config.go, config_apply_test.go, bootstrap_guard_test.go, internal/config/yaml_config.go, docs/reference/configuration.md). I read each seam against main's intent, #7616's urlHostname rewrite and the scrubURLCredentials on the bootstrap error print and #7650's TestMain guard among them, and changed nothing.

Range-diff against b1c84c1f5: = for commits 2 to 5; commit 1 differs by the manifest rows only, the interdiff outside that file is empty.

Gates on d3dd0bc43, all green: make test-go (121 packages ok, 0 failures, with the dolt CLI on PATH), go test ./scripts/ (the shard-manifest checks among them), golangci-lint 0 issues and go vet (both under -tags gms_pure_go; this machine has no ICU headers for the cgo regex path main pulls in), gofmt, tools/bazel/go_srcs.py --check, the body's -run set, internal/config, the four TestEmbeddedGitDataRef* end-to-end tests with BEADS_TEST_EMBEDDED_DOLT=1, the CGO_ENABLED=0 test-binary compile, git diff --check, doc freshness. The base's CI on the new head is running as I write this.

@bee-ghosttrack bee-ghosttrack left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental re-review of the rebase b1c84c1f5 → d3dd0bc43 (onto ce70b261c), read as a range-diff against my 10-14 approval rather than as a fresh pass.

Patch identity. git range-diff <oldbase>..b1c84c1f5 ce70b261c..d3dd0bc43: commits 2–5 are =; commit 1 is ! and the interdiff is exactly the four TestEmbeddedGitDataRef* rows of .github/scripts/embedded-cmd-test-shards.txt moving from shards 61–64 to 56–59. Same file set on both sides of commit 1; nothing else moved. That matches the description.

Conflict resolution. scripts/ci/gen_embedded_cmd_shard_manifest.py 100 --weights=duration --check exits 0 on d3dd0bc43; the manifest diff against main is the four added rows and nothing else.

Behaviour on the new base. BEADS_TEST_EMBEDDED_DOLT=1 go test ./cmd/bd/ -run TestEmbeddedGitDataRef -count=1 -v: 4/4 PASS, 0 SKIP (129s) — so the ref-following bootstrap paths are fine over #6943's migration 0070 / write fence.

go test ./scripts/ -v -count=1: 294 pass / 11 skip / 1 fail. The one failure, TestPRRunsGoTestsBazelSkips, wants stdversion in tools/nogo/analyzers.bzl VET_PASSES; it fails identically on origin/main ce70b261c and this PR touches neither the list nor the test — it is my local go1.27.1 (go.mod toolchain is 1.26.7) adding stdversion to go test's vet set, not this change. Noting it here so nobody re-derives it; it belongs in a separate main fix.

Approving again on d3dd0bc43.

@risperdal

Copy link
Copy Markdown

Adding a use case this unblocks.

We want to run bd inside Claude Code cloud sessions (Anthropic-hosted VMs). The VM holds no git credentials. All git traffic goes through Anthropic's GitHub proxy, and that proxy rejects every push that isn't to a branch (tags included). In practice bd works read-only there: pulling from refs/dolt/data is fine, but pushing to it is refused. The proxy doesn't care which branch, though, so a dedicated branch would work:

bd dolt remote add origin https://github.com/<org>/<repo>.git --ref refs/heads/beads-data
bd config set sync.remote-ref refs/heads/beads-data

#6521 already carries the ref in storage, but without this PR there's no CLI path to point the remote at a branch.

Would be great to see this land.

@risperdal

Copy link
Copy Markdown

@bee-ghosttrack @sdghbtn any chance this could get merged soon? It's already approved, just needs a rebase again. We'd really like to use bd from Claude Code cloud agents and this is the only thing blocking writes there. Happy to test once it's rebased.

@bee-ghosttrack

Copy link
Copy Markdown
Collaborator

Status from my side (bee, 2026-11-05): my approval stands at the current head d3dd0bc43b, and the checks on it are green (29 pass). What is in the way is only that the branch is CONFLICTING against main again since your 2026-10-23 note; GitHub will not merge a conflicting head. So the one thing on your side is the rebase onto current main (or a merge of main into the branch); when it lands I will diff the new head against d3dd0bc43b and, if the content is unchanged, renew the approval the same day. Merging itself is a maintainer action; I review but have no write access.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature New user-facing capability priority/p2 Medium: real problem with lower urgency

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bd dolt remote add / bd bootstrap cannot use a git remote whose Dolt data is on a branch (no --ref)

3 participants