Replies: 1 comment 2 replies
-
The progress reporting was added because the initial migration implementation was very slow. See #6367:
🤔 this is interesting - my library of a similar size took 2s to migrate - we considered running migrations separately, but decided to perform them automatically because the operation was assumed to be very quick, even for large libraries. Would you mind making a copy of your library, re-running this migration and measuring the time it takes to run? I want to understand why is it taking so long. $ sqlite3 <your-library-copy> "DELETE FROM migrations WHERE name == 'multi_genre_field'"
$ sqlite3 <your-library-copy> "UPDATE items SET genres = NULL"
$ sqlite3 <your-library-copy> "UPDATE albums SET genres = NULL"
$ time beet --library=<your-library-copy> help |
Beta Was this translation helpful? Give feedback.
-
|
Hm, that was a lot faster: The earlier run had additional migrations logged -- I didn't keep the full output, but there was at least one extra step last time: Possibly I had an older migration pending too? The library database is on a SSD but the media itself is on an HDD, so anything involving walking the actual files would likely take a lot longer, if that was required? |
Beta Was this translation helpful? Give feedback.
-
|
Interesting. The
Thankfully, not. These migrations don't even loop across items or albums - each of them queries and mutates data in the DB directly, thus it's very quick. For example, the def _migrate_field(self, model_cls: type[Model], field: str) -> None:
table = model_cls._table
with self.db.transaction() as tx:
rows = tx.query(
f"""
SELECT id, {field}
FROM {table}
WHERE {field} IS NOT NULL
"""
)
total = len(rows)
to_migrate = [r for r in rows if r[field] and os.path.isabs(r[field])]
if not to_migrate:
return
ui.print_(f"Migrating {field} for {total} {table}...")
with self.db.transaction() as tx:
tx.mutate_many(
f"UPDATE {table} SET {field} = ? WHERE id = ?",
[
# Convert to bytes in case a user has manually set paths to
# a TEXT value
(normalize_path_for_db(os.fsencode(r[field])), r["id"])
for r in to_migrate
],
)
ui.print_(
f"Migration complete: {len(to_migrate)} of {total} {table} updated"
) |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I vaguely remembered reading about the database migration in the release notes, though I only today needed to reach for
beetand was surprised that calling--helpstarted a migration:This isn't ideal when you're connected to a remote server over a flaky connection! It's unclear what would happen if interrupted -- with the progress reporting I'm assuming this wasn't an atomic operation. It took about 4-5 minutes to run in the end, after which I got the help I requested:
Ultimately this was a one and done, but for future migrations it would be nice to make running the migration an explicit action, e.g.
...or at least exclude no-op commands like
--helpfrom starting it.Beta Was this translation helpful? Give feedback.
All reactions