Repository navigation
Conversation
By default, the import script recreates the database schema, which includes creating new sequences with zero values. This results in the lazy crates.io developer occasionally receiving obscure errors when inserting records into tables that use sequences, often not on the first or second insert due to IDs in the database dump not always being continuous. Rather than dumping the real sequence values from the database, we can just recreate them based on the maximum ID in each table. Works well enough, and means we don't have to tinker with the export script or ship extra data. This commit only configures the database tables that actually include data in the database dump. There are other sequences, but since those tables won't have data imported, it doesn't matter if they remain zero after import.
LawnGnome
force-pushed
the
db-dump-sequences
branch
from
December 13, 2024 23:08
e279712 to
bd4780f
Compare
Turbo87
reviewed
Dec 14, 2024
| path = "public" | ||
| [categories.sequence] | ||
| column = "id" | ||
| name = "categories_id_seq" |
Member
There was a problem hiding this comment.
instead of manually declaring them here, would it be possible to derive them from the database schema in some way?
Contributor
There was a problem hiding this comment.
This is doable as follows, which is modified from https://stackoverflow.com/a/55414721:
select tbl.relname as table_name,
col.attname as column_name,
s.relname as sequence_name
from pg_class s
join pg_namespace sn on sn.oid = s.relnamespace
join pg_depend d on d.refobjid = s.oid and d.refclassid='pg_class'::regclass
join pg_attrdef ad on ad.oid = d.objid and d.classid = 'pg_attrdef'::regclass
join pg_attribute col on col.attrelid = ad.adrelid and col.attnum = ad.adnum
join pg_class tbl on tbl.oid = ad.adrelid
join pg_namespace ts on ts.oid = tbl.relnamespace
where s.relkind = 'S'
-- and s.relname = 'your_sequence_name_her'
and d.deptype in ('a', 'n');From the result, you should see something similar to the following:
table_name | column_name | sequence_name
-----------------------+-------------+------------------------------
api_tokens | id | api_tokens_id_seq
background_jobs | id | background_jobs_id_seq
categories | id | categories_id_seq
crates | id | packages_id_seq
deleted_crates | id | deleted_crates_id_seq
dependencies | id | dependencies_id_seq
emails | id | emails_id_seq
keywords | id | keywords_id_seq
teams | id | teams_id_seq
users | id | users_id_seq
version_owner_actions | id | version_owner_actions_id_seq
versions | id | versions_id_seq
(12 rows)
| 1 | ||
| ) | ||
| ); | ||
| {% endif %} |
Member
There was a problem hiding this comment.
I think this needs an update of the corresponding test snapshot :)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By default, the import script recreates the database schema, which includes creating new sequences with zero values. This results in the lazy crates.io developer occasionally receiving obscure errors when inserting records into tables that use sequences, often not on the first or second insert due to IDs in the database dump not always being continuous.
Rather than dumping the real sequence values from the database, we can just recreate them based on the maximum ID in each table. Works well enough, and means we don't have to tinker with the export script or ship extra data.
This commit only configures the database tables that actually include data in the database dump. There are other sequences, but since those tables won't have data imported, it doesn't matter if they remain zero after import.