Skip to content

Commit 8840f90

Browse files
HugoFaraHugoFara
authored
test(db): provision the test database the way production does (#254)
The test database was missing three tables and carried the wrong type for languages.LgID, so whole areas of the suite exercised nothing. setup_test_db.php applied baseline.sql and then marked every migration as applied without running it, on the premise that "the baseline already includes all table structures from migrations". That premise is false for anything added after the baseline was last regenerated: `books`, `local_dictionaries` and `local_dictionary_entries` appear only in migrations, so those tables never existed here while their migrations were recorded as applied. Tests touching them skipped in silence, locally and on CI -- 35 of the 44 in DictionaryFacadeTest, including the regression tests for (#250). Skipping 20251221_120000_add_inter_table_foreign_keys.sql also left languages.LgID at the baseline's tinyint(3). Production widens it to int(11) there, and later migrations declare FK columns as int(11) to match. Creating such a table against a tinyint parent fails with errno 150, which FOREIGN_KEY_CHECKS=0 does not suppress -- so even had the migrations run, local_dictionaries would not have been created. Migrations now actually run, in order, statement by statement, skipping any already recorded and tolerating per-statement failures exactly as Migrations::update() does (legacy migrations reference tables the modern baseline no longer has). Three further adjustments were needed to get a schema that matches production: - Finish the column widening under modern table names. The FK migration targets `textitems2` and `newsfeeds`, which the baseline creates as `word_occurrences` and `news_feeds`, so its statements for them no-op and left those columns too narrow for their foreign keys. - Add the FK constraints unconditionally, skipping ones already present, rather than gating the whole block on whether the FK migration was recorded. That gate meant a database whose migrations had run got no FK constraints at all. - Clear orphaned rows before adding a constraint. The main suite drops every foreign key (Migrations::dropAllForeignKeys, via the restore and migration paths) and can leave children whose parent is gone; this script also runs non-destructively before an integration run, so without the cleanup the constraint cannot be re-added. A fresh database goes from 16 tables to 20, with languages.LgID at int(11) and 28 foreign keys instead of 14. The integration suite runs 12 more tests than before (skips 25 -> 13, assertions 356 -> 384), passing both on a fresh database and on one left dirty by a full suite run. The 9085-test suite is unchanged. Note that production was never affected: Migrations::checkAndUpdate applies baseline.sql and then runs every pending migration in order, so LgID is already int(11) by the time the dictionary migration runs.
1 parent 5059847 commit 8840f90

1 file changed

Lines changed: 207 additions & 74 deletions

File tree

‎tests/setup_test_db.php‎

Lines changed: 207 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,13 @@
2525
namespace Lwt\Tests;
2626

2727
use Lwt\Shared\Infrastructure\Bootstrap\EnvLoader;
28+
use Lwt\Shared\Infrastructure\Database\SqlFileParser;
2829

2930
// Load environment configuration
3031
require_once __DIR__ . '/../src/Shared/Infrastructure/Bootstrap/EnvLoader.php';
32+
// Autoloader for SqlFileParser, so migrations are split into statements exactly
33+
// the way Migrations::update() splits them in production.
34+
require_once __DIR__ . '/../vendor/autoload.php';
3135

3236
// Parse command line arguments
3337
$drop = in_array('--drop', $argv ?? []);
@@ -249,27 +253,78 @@ function hasForeignKeys(\mysqli $conn, string $dbName): bool
249253
}
250254
sort($migrationFiles);
251255

252-
// The baseline schema already includes all table structures from migrations.
253-
// We need to:
254-
// 1. Mark most migrations as "applied" (since baseline incorporates their changes)
255-
// 2. Actually run specific migrations that need explicit execution:
256-
// - FK migration (adds inter-table foreign keys)
257-
// - Column defaults migration (mysqli_multi_query doesn't handle DEFAULT '' correctly)
258-
$fkMigration = '20251221_120000_add_inter_table_foreign_keys.sql';
256+
// Production (Migrations::checkAndUpdate) applies baseline.sql and then runs
257+
// every pending migration in order, tolerating per-statement failures. This
258+
// script has to do the same, or the test database drifts from what users
259+
// actually have.
260+
//
261+
// It used to mark migrations as applied without running them, on the premise
262+
// that "the baseline already includes all table structures". That premise is
263+
// false for anything added after the baseline was last regenerated — `books`,
264+
// `local_dictionaries` and `local_dictionary_entries` are all absent from it —
265+
// so those tables never existed here while their migrations were recorded as
266+
// applied. Every test touching them then skipped silently, locally and on CI.
267+
//
268+
// Skipping the FK migration also left languages.LgID as tinyint(3), because
269+
// that migration is what widens it to int(11); the manual FK list below never
270+
// carried the type changes. Any later migration with an FK to languages(LgID)
271+
// then failed with errno 150 ("Foreign key constraint is incorrectly formed"),
272+
// which is not suppressed by FOREIGN_KEY_CHECKS=0.
259273
$columnDefaultsMigration = '20260107_120000_add_language_column_defaults.sql';
274+
$fkMigration = '20251221_120000_add_inter_table_foreign_keys.sql';
275+
276+
// Only pending migrations are run, as in production: this script is also
277+
// invoked non-destructively before each integration run, and re-executing
278+
// every migration each time would be both slow and unsafe for any migration
279+
// that moves data rather than just shaping schema.
280+
$alreadyApplied = [];
281+
$result = mysqli_query($conn, "SELECT filename FROM _migrations");
282+
if ($result) {
283+
while ($row = mysqli_fetch_assoc($result)) {
284+
$alreadyApplied[] = $row['filename'];
285+
}
286+
mysqli_free_result($result);
287+
}
260288

261-
output("Marking migrations as applied (baseline includes these changes)...\n", $quiet);
289+
output("Applying migrations...\n", $quiet);
290+
mysqli_query($conn, "SET FOREIGN_KEY_CHECKS = 0");
291+
$migrationsRun = 0;
292+
$statementFailures = 0;
262293
foreach ($migrationFiles as $migrationFile) {
263294
$filename = basename($migrationFile);
264295

265-
// Skip migrations that need to be run explicitly
266-
if ($filename === $fkMigration || $filename === $columnDefaultsMigration) {
296+
// Applied explicitly further down: applying baseline.sql through
297+
// mysqli_multi_query drops the DEFAULT '' clauses this migration relies on.
298+
if ($filename === $columnDefaultsMigration) {
267299
continue;
268300
}
269301

302+
if (in_array($filename, $alreadyApplied, true)) {
303+
continue;
304+
}
305+
306+
foreach (SqlFileParser::parseFile($migrationFile) as $statement) {
307+
if (trim($statement) === '') {
308+
continue;
309+
}
310+
if (!@mysqli_query($conn, $statement)) {
311+
// Match production, which logs a failed statement and carries on:
312+
// legacy migrations reference tables the modern baseline no longer
313+
// has, and those failures are expected.
314+
$statementFailures++;
315+
}
316+
}
317+
270318
$escapedFilename = mysqli_real_escape_string($conn, $filename);
271319
mysqli_query($conn, "INSERT IGNORE INTO _migrations (filename, applied_at) VALUES ('$escapedFilename', NOW())");
320+
$migrationsRun++;
272321
}
322+
mysqli_query($conn, "SET FOREIGN_KEY_CHECKS = 1");
323+
output(
324+
"Ran $migrationsRun migration(s)"
325+
. ($statementFailures > 0 ? " ($statementFailures statement(s) skipped)" : '') . ".\n",
326+
$quiet
327+
);
273328

274329
// Get applied migrations (to check if FK migration was already applied)
275330
$appliedMigrations = [];
@@ -281,77 +336,155 @@ function hasForeignKeys(\mysqli $conn, string $dbName): bool
281336
mysqli_free_result($result);
282337
}
283338

284-
// Apply FK constraints directly (baseline has matching column types)
285-
// The FK migration file modifies column types which breaks fresh installs
286-
// So we apply FK constraints directly here
339+
// Complete the inter-table foreign keys.
340+
//
341+
// The FK migration above only gets us part of the way: it was written against
342+
// the legacy `textitems2` table, which the modern baseline creates as
343+
// `word_occurrences`, so its statements for that table fail and the FKs the
344+
// integration tests rely on never appear. The list below names the modern
345+
// tables. It runs unconditionally because adding an existing constraint is
346+
// reported as a duplicate and ignored, so it is safe to re-apply.
287347
$appliedCount = 0;
288-
if (!in_array($fkMigration, $appliedMigrations)) {
289-
output("Applying foreign key constraints...\n", $quiet);
290-
291-
// FK constraints to add (baseline already has matching column types)
292-
$fkConstraints = [
293-
// Language references
294-
"ALTER TABLE texts ADD CONSTRAINT fk_texts_language " .
295-
"FOREIGN KEY (TxLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
296-
"ALTER TABLE words ADD CONSTRAINT fk_words_language " .
297-
"FOREIGN KEY (WoLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
298-
"ALTER TABLE sentences ADD CONSTRAINT fk_sentences_language " .
299-
"FOREIGN KEY (SeLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
300-
"ALTER TABLE news_feeds ADD CONSTRAINT fk_news_feeds_language " .
301-
"FOREIGN KEY (NfLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
302-
// Text references
303-
"ALTER TABLE sentences ADD CONSTRAINT fk_sentences_text " .
304-
"FOREIGN KEY (SeTxID) REFERENCES texts(TxID) ON DELETE CASCADE",
305-
"ALTER TABLE word_occurrences ADD CONSTRAINT fk_word_occurrences_text " .
306-
"FOREIGN KEY (Ti2TxID) REFERENCES texts(TxID) ON DELETE CASCADE",
307-
"ALTER TABLE text_tag_map ADD CONSTRAINT fk_text_tag_map_text " .
308-
"FOREIGN KEY (TtTxID) REFERENCES texts(TxID) ON DELETE CASCADE",
309-
// Sentence reference
310-
"ALTER TABLE word_occurrences ADD CONSTRAINT fk_word_occurrences_sentence " .
311-
"FOREIGN KEY (Ti2SeID) REFERENCES sentences(SeID) ON DELETE CASCADE",
312-
// Word reference (SET NULL for unknown words)
313-
"ALTER TABLE word_occurrences MODIFY COLUMN Ti2WoID mediumint(8) unsigned DEFAULT NULL",
314-
"ALTER TABLE word_occurrences ADD CONSTRAINT fk_word_occurrences_word " .
315-
"FOREIGN KEY (Ti2WoID) REFERENCES words(WoID) ON DELETE SET NULL",
316-
// Word tags
317-
"ALTER TABLE word_tag_map ADD CONSTRAINT fk_word_tag_map_word " .
318-
"FOREIGN KEY (WtWoID) REFERENCES words(WoID) ON DELETE CASCADE",
319-
"ALTER TABLE word_tag_map ADD CONSTRAINT fk_word_tag_map_tag " .
320-
"FOREIGN KEY (WtTgID) REFERENCES tags(TgID) ON DELETE CASCADE",
321-
// Text tags
322-
"ALTER TABLE text_tag_map ADD CONSTRAINT fk_text_tag_map_text_tag " .
323-
"FOREIGN KEY (TtT2ID) REFERENCES text_tags(T2ID) ON DELETE CASCADE",
324-
// Feed links
325-
"ALTER TABLE feed_links ADD CONSTRAINT fk_feed_links_newsfeed " .
326-
"FOREIGN KEY (FlNfID) REFERENCES news_feeds(NfID) ON DELETE CASCADE",
327-
];
348+
output("Applying foreign key constraints...\n", $quiet);
349+
350+
// Widen the referencing columns first. The FK migration widens the
351+
// referenced keys (languages.LgID, texts.TxID, words.WoID, sentences.SeID,
352+
// tags.TgID) to int(11), but its statements for `textitems2` and
353+
// `newsfeeds` no-op against a modern baseline that names those tables
354+
// `word_occurrences` and `news_feeds`. Their columns are left at the
355+
// baseline's narrower widths, and an FK between mismatched integer types
356+
// fails with errno 150. These MODIFYs finish the job under the new names.
357+
$columnWidening = [
358+
"ALTER TABLE news_feeds MODIFY COLUMN NfID int(11) unsigned NOT NULL AUTO_INCREMENT",
359+
"ALTER TABLE news_feeds MODIFY COLUMN NfLgID int(11) unsigned NOT NULL",
360+
"ALTER TABLE feed_links MODIFY COLUMN FlNfID int(11) unsigned NOT NULL",
361+
"ALTER TABLE word_occurrences MODIFY COLUMN Ti2TxID int(11) unsigned NOT NULL",
362+
"ALTER TABLE word_occurrences MODIFY COLUMN Ti2SeID int(11) unsigned NOT NULL",
363+
"ALTER TABLE word_occurrences MODIFY COLUMN Ti2WoID int(11) unsigned DEFAULT NULL",
364+
"ALTER TABLE word_tag_map MODIFY COLUMN WtWoID int(11) unsigned NOT NULL",
365+
"ALTER TABLE word_tag_map MODIFY COLUMN WtTgID int(11) unsigned NOT NULL",
366+
"ALTER TABLE text_tags MODIFY COLUMN T2ID int(11) unsigned NOT NULL AUTO_INCREMENT",
367+
"ALTER TABLE text_tag_map MODIFY COLUMN TtTxID int(11) unsigned NOT NULL",
368+
"ALTER TABLE text_tag_map MODIFY COLUMN TtT2ID int(11) unsigned NOT NULL",
369+
];
370+
foreach ($columnWidening as $sql) {
371+
@mysqli_query($conn, $sql);
372+
}
373+
374+
// FK constraints to add (column types now match on both sides)
375+
$fkConstraints = [
376+
// Language references
377+
"ALTER TABLE texts ADD CONSTRAINT fk_texts_language " .
378+
"FOREIGN KEY (TxLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
379+
"ALTER TABLE words ADD CONSTRAINT fk_words_language " .
380+
"FOREIGN KEY (WoLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
381+
"ALTER TABLE sentences ADD CONSTRAINT fk_sentences_language " .
382+
"FOREIGN KEY (SeLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
383+
"ALTER TABLE news_feeds ADD CONSTRAINT fk_news_feeds_language " .
384+
"FOREIGN KEY (NfLgID) REFERENCES languages(LgID) ON DELETE CASCADE",
385+
// Text references
386+
"ALTER TABLE sentences ADD CONSTRAINT fk_sentences_text " .
387+
"FOREIGN KEY (SeTxID) REFERENCES texts(TxID) ON DELETE CASCADE",
388+
"ALTER TABLE word_occurrences ADD CONSTRAINT fk_word_occurrences_text " .
389+
"FOREIGN KEY (Ti2TxID) REFERENCES texts(TxID) ON DELETE CASCADE",
390+
"ALTER TABLE text_tag_map ADD CONSTRAINT fk_text_tag_map_text " .
391+
"FOREIGN KEY (TtTxID) REFERENCES texts(TxID) ON DELETE CASCADE",
392+
// Sentence reference
393+
"ALTER TABLE word_occurrences ADD CONSTRAINT fk_word_occurrences_sentence " .
394+
"FOREIGN KEY (Ti2SeID) REFERENCES sentences(SeID) ON DELETE CASCADE",
395+
// Word reference (SET NULL for unknown words)
396+
// (Ti2WoID is made nullable and widened to match words.WoID in the
397+
// column-widening step above.)
398+
"ALTER TABLE word_occurrences ADD CONSTRAINT fk_word_occurrences_word " .
399+
"FOREIGN KEY (Ti2WoID) REFERENCES words(WoID) ON DELETE SET NULL",
400+
// Word tags
401+
"ALTER TABLE word_tag_map ADD CONSTRAINT fk_word_tag_map_word " .
402+
"FOREIGN KEY (WtWoID) REFERENCES words(WoID) ON DELETE CASCADE",
403+
"ALTER TABLE word_tag_map ADD CONSTRAINT fk_word_tag_map_tag " .
404+
"FOREIGN KEY (WtTgID) REFERENCES tags(TgID) ON DELETE CASCADE",
405+
// Text tags
406+
"ALTER TABLE text_tag_map ADD CONSTRAINT fk_text_tag_map_text_tag " .
407+
"FOREIGN KEY (TtT2ID) REFERENCES text_tags(T2ID) ON DELETE CASCADE",
408+
// Feed links
409+
"ALTER TABLE feed_links ADD CONSTRAINT fk_feed_links_newsfeed " .
410+
"FOREIGN KEY (FlNfID) REFERENCES news_feeds(NfID) ON DELETE CASCADE",
411+
];
412+
413+
// Constraints already on the database are left alone. This script also runs
414+
// non-destructively before each integration run, when the database holds
415+
// rows from a previous suite; re-adding an existing constraint would then
416+
// fail against test data that predates it (an orphaned word_tag_map row is
417+
// enough) and silently leave the constraint dropped.
418+
$existingConstraints = [];
419+
$constraintRows = mysqli_query(
420+
$conn,
421+
"SELECT CONSTRAINT_NAME FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
422+
WHERE TABLE_SCHEMA = '$testDbName' AND CONSTRAINT_TYPE = 'FOREIGN KEY'"
423+
);
424+
if ($constraintRows) {
425+
while ($row = mysqli_fetch_assoc($constraintRows)) {
426+
$existingConstraints[] = (string) $row['CONSTRAINT_NAME'];
427+
}
428+
mysqli_free_result($constraintRows);
429+
}
430+
431+
// Clear rows that would violate a constraint before adding it. The main
432+
// suite drops every foreign key (Migrations::dropAllForeignKeys, reached
433+
// through the restore and migration paths) and can leave children behind
434+
// whose parent is gone. This script also runs non-destructively before an
435+
// integration run, so without this the constraint cannot be re-added and
436+
// the cascade tests that depend on it fail — or, as before, skip in
437+
// silence.
438+
$orphanCleanup = [
439+
"DELETE c FROM word_tag_map c LEFT JOIN words p ON c.WtWoID = p.WoID WHERE p.WoID IS NULL",
440+
"DELETE c FROM word_tag_map c LEFT JOIN tags p ON c.WtTgID = p.TgID WHERE p.TgID IS NULL",
441+
"DELETE c FROM text_tag_map c LEFT JOIN texts p ON c.TtTxID = p.TxID WHERE p.TxID IS NULL",
442+
"DELETE c FROM text_tag_map c LEFT JOIN text_tags p ON c.TtT2ID = p.T2ID WHERE p.T2ID IS NULL",
443+
"DELETE c FROM word_occurrences c LEFT JOIN texts p ON c.Ti2TxID = p.TxID WHERE p.TxID IS NULL",
444+
"DELETE c FROM word_occurrences c LEFT JOIN sentences p ON c.Ti2SeID = p.SeID WHERE p.SeID IS NULL",
445+
"UPDATE word_occurrences c LEFT JOIN words p ON c.Ti2WoID = p.WoID
446+
SET c.Ti2WoID = NULL WHERE c.Ti2WoID IS NOT NULL AND p.WoID IS NULL",
447+
"DELETE c FROM sentences c LEFT JOIN texts p ON c.SeTxID = p.TxID WHERE p.TxID IS NULL",
448+
"DELETE c FROM feed_links c LEFT JOIN news_feeds p ON c.FlNfID = p.NfID WHERE p.NfID IS NULL",
449+
"DELETE c FROM texts c LEFT JOIN languages p ON c.TxLgID = p.LgID WHERE p.LgID IS NULL",
450+
"DELETE c FROM words c LEFT JOIN languages p ON c.WoLgID = p.LgID WHERE p.LgID IS NULL",
451+
"DELETE c FROM sentences c LEFT JOIN languages p ON c.SeLgID = p.LgID WHERE p.LgID IS NULL",
452+
"DELETE c FROM news_feeds c LEFT JOIN languages p ON c.NfLgID = p.LgID WHERE p.LgID IS NULL",
453+
];
454+
foreach ($orphanCleanup as $sql) {
455+
@mysqli_query($conn, $sql);
456+
}
328457

329-
$fkCount = 0;
330-
$fkErrors = 0;
331-
foreach ($fkConstraints as $sql) {
332-
if (@mysqli_query($conn, $sql)) {
333-
$fkCount++;
334-
} else {
335-
$error = mysqli_error($conn);
336-
// Ignore "duplicate key" errors (constraint already exists)
337-
if (strpos($error, 'Duplicate') === false && strpos($error, 'already exists') === false) {
338-
$fkErrors++;
339-
if (!$quiet) {
340-
fwrite(STDERR, " Warning: " . $error . "\n");
341-
}
458+
$fkCount = 0;
459+
$fkErrors = 0;
460+
foreach ($fkConstraints as $sql) {
461+
if (
462+
preg_match('/ADD CONSTRAINT (\w+)/', $sql, $match) === 1
463+
&& in_array($match[1], $existingConstraints, true)
464+
) {
465+
continue;
466+
}
467+
468+
if (@mysqli_query($conn, $sql)) {
469+
$fkCount++;
470+
} else {
471+
$error = mysqli_error($conn);
472+
// Ignore "duplicate key" errors (constraint already exists)
473+
if (strpos($error, 'Duplicate') === false && strpos($error, 'already exists') === false) {
474+
$fkErrors++;
475+
if (!$quiet) {
476+
fwrite(STDERR, " Warning: " . $error . "\n");
342477
}
343478
}
344479
}
480+
}
345481

346-
// Record migration as applied
347-
$escapedFilename = mysqli_real_escape_string($conn, $fkMigration);
348-
mysqli_query($conn, "INSERT IGNORE INTO _migrations (filename, applied_at) VALUES ('$escapedFilename', NOW())");
482+
// Record migration as applied
483+
$escapedFilename = mysqli_real_escape_string($conn, $fkMigration);
484+
mysqli_query($conn, "INSERT IGNORE INTO _migrations (filename, applied_at) VALUES ('$escapedFilename', NOW())");
349485

350-
output("Applied $fkCount FK constraint(s)" . ($fkErrors > 0 ? " ($fkErrors warnings)" : "") . ".\n", $quiet);
351-
$appliedCount = 1;
352-
} else {
353-
output("FK constraints already applied.\n", $quiet);
354-
}
486+
output("Applied $fkCount FK constraint(s)" . ($fkErrors > 0 ? " ($fkErrors warnings)" : "") . ".\n", $quiet);
487+
$appliedCount = 1;
355488

356489
// Apply column defaults migration (mysqli_multi_query doesn't handle DEFAULT '' correctly in baseline.sql)
357490
if (!in_array($columnDefaultsMigration, $appliedMigrations)) {

0 commit comments

Comments
 (0)