Skip to content

[Misc] Document data migrations and the document-queueing pattern - #18

Draft
manuelleduc wants to merge 3 commits into
masterfrom
okf-data-migrations
Draft

[Misc] Document data migrations and the document-queueing pattern#18
manuelleduc wants to merge 3 commits into
masterfrom
okf-data-migrations

Conversation

@manuelleduc

@manuelleduc manuelleduc commented Jul 22, 2026

Copy link
Copy Markdown

Adds one OKF topic the map was missing: how a data migration is shaped, and the three rules that
otherwise make a document-queueing migration a silent no-op. Learned while validating the
XWIKI-20699 migration with real 17.10.9 → 18.7.0-SNAPSHOT upgrades on the supported database engines.

okf/architecture/data-migrations.md

  • The queueing pattern — AbstractDocumentsMigration only writes rows into the async indexing
    queue and a separate TaskConsumer performs the change later, so the migration reporting success
    does not mean the data changed and the two halves fail independently.
  • Pass "", never null, as the locale: LocaleUtils.toLocale(null) returns null rather than
    throwing, so every document resolves to Optional.empty() and nothing is queued.
  • A TaskConsumer comparing resolved references needs @Named("current"), or the default resolver
    falls back to the default wiki and every subwiki is silently skipped — invisible on the main wiki.
  • Guard possibly-empty strings in HQL with length(x) > 0, not <> '', because Oracle stores the
    empty string as NULL.
  • A closing paragraph on validation: unit tests are structurally blind to the first two rules (a
    mocked resolver behaves the same whichever implementation is injected), so it takes a real
    FROM → TO upgrade, run on a subwiki as well as the main wiki, with the task queue drained before
    asserting, and on Oracle.

Scope

Deliberately kept to one file. The engine versions are marked volatile with a verify: recipe
pointing at the Release Notes and the Database Support Strategy page; only the policy is inline.
Harness-specific traps (building the distribution, readiness probing, extension.repositories,
patching the shipped Hibernate config) are left out — they are procedure, already covered by the
xwiki-build skill, or generic DBA trivia that would cost tokens in every session that loads this
file.

Grounding

The architecture claims are cited to the platform sources they come from and were re-read against
master while writing. The engine-specific behaviour was observed on a running instance.

Index and the injected map updated; plugin version bumped 1.1.7 → 1.1.8 across the manifests;
node scripts/validate.mjs and claude plugin validate ./xwiki pass.

@manuelleduc
manuelleduc marked this pull request as draft July 23, 2026 12:10
* Add okf/architecture/data-migrations.md: the queueing pattern
  (AbstractDocumentsMigration queues, a TaskConsumer changes) and the three
  rules that otherwise make such a pair a silent no-op — the empty-string
  locale, the "current" resolver hint, and length()>0 in HQL for Oracle.
* Fold the validation guidance into the same file as one paragraph: real
  upgrade, subwiki, drain the queue, Oracle.
* Update okf/index.md and the injected map, and bump the plugin version.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vmassol
vmassol force-pushed the okf-data-migrations branch from dfba0e1 to 579c767 Compare August 23, 2026 17:41
@vmassol vmassol changed the title [Misc] Document data migrations and their cross-database validation [Misc] Document data migrations and the document-queueing pattern Aug 23, 2026
@vmassol

vmassol commented Aug 23, 2026

Copy link
Copy Markdown
Member

@manuelleduc I've updated it a bit to conform to best practices in the OKF. Please let me know what you want to do about this PR. Thx!

vmassol and others added 2 commits September 1, 2026 16:27
* Five manifests carry one version, so every concurrent pull request conflicted with every other one
  on those same five lines, over something that was never part of either change. The release is now
  cut on master after the merge by scripts/release.mjs, and scripts/validate.mjs fails a branch that
  touches a version field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts:
#	xwiki/instructions/xwiki-org.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants