Skip to content

feat: add postgres section for retire-dataset dag - #109

Merged
tombrooks248 merged 1 commit into
mainfrom
feat/add-postgres-section-for-retire-dataset
Oct 7, 2026
Merged

tombrooks248 merged 1 commit into
mainfrom
feat/add-postgres-section-for-retire-dataset

Conversation

@tombrooks248

@tombrooks248 tombrooks248 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Description

Adds a postgres task to the retire-dataset DAG. It runs after discover, alongside files, and runs the Postgres loader image with the command ./retire.sh (digital-land/digital-land-postgres#31), passing DATASET_NAME and the same DRY_RUN value as files. The loader removes the dataset's rows from entity, old_entity and entity_subdivided, or only counts them on a dry run. The DAG docs now describe the task and how removed rows can be loaded again.

Why

Part 2 of digital-land/digital-land-builder#64. Removing a dataset's files leaves its entities in Postgres, so its entity pages and API results stay live. This is what takes a retired dataset off the site.

Related Tickets & Documents

QA Instructions, Screenshots, Recordings

  • Deployed this branch to development and ran retire-dataset with retire-dataset-test (not in the specification) as a dry run (run manual__2026-10-07T13:43:58):
    • guard passed and discover logged no user-facing files found for retire-dataset-test in development-collection-data.
    • postgres started development-sqlite-ingestion-task with command: ['./retire.sh'], DATASET_NAME: 'retire-dataset-test' and DRY_RUN: 'true', all rendered as plain strings.
    • The container logged Retiring retire-dataset-test rows from postgres (DRY_RUN=true) and then would remove 0 retire-dataset-test rows from each of entity, old_entity and entity_subdivided. It exited 0, which shows the task reaches the database with the task definition's credentials and removes nothing on a dry run.
  • The loader's retire mode (feat: enable retire dataset function digital-land-postgres#31) was tested separately against a local PostGIS database. A dry run changed nothing, and DRY_RUN=false removed only the given dataset's rows. It is deployed to every environment, including production.

Added/updated tests?

We encourage you to keep the code coverage percentage at 80% and above. Please refer to the Digital Land Testing Guidance for more information.

  • Yes
  • No, and this is why: please replace this line with details on why tests
    have not been included
  • I need help with writing tests

[optional] Are there any post deployment tasks we need to perform?

[optional] Are there any dependencies on other PRs or Work?

@tombrooks248
tombrooks248 merged commit dbf1f19 into main Oct 7, 2026
5 checks passed

This branch was successfully deployed

1 active deployment
development — 33ac123f Deployed Oct 7, 2026 by tombrooks248 via deploy (development) #706
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant