Skip to content

Commit

Permalink
Don't include facts from ended resources.
Browse files Browse the repository at this point in the history
  • Loading branch information
cjohns-scottlogic committed Dec 2, 2024
1 parent 2f9000e commit b453b4a
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion digital_land/package/datasetparquet.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,10 +190,11 @@ def load_entities(self):
# If there are still matches then pick the first resource (and fact, just to make sure)
query = f"""
SELECT {fields_str} FROM (
SELECT {fields_str}, CASE WHEN resource_csv."end-date" IS NULL THEN '2999-12-31' ELSE resource_csv."end-date" END AS resource_end_date
SELECT {fields_str}, resource_csv."end-date" AS resource_end_date
FROM temp_table
LEFT JOIN read_csv_auto('collection/resource.csv') resource_csv
ON temp_table.resource = resource_csv.resource
WHERE resource_end_date IS NULL OR resource_end_date > current_date
QUALIFY ROW_NUMBER() OVER (
PARTITION BY entity, field
ORDER BY priority, "entry-date" DESC, "entry-number" DESC, resource_end_date DESC, temp_table.resource, fact
Expand Down

0 comments on commit b453b4a

Please sign in to comment.