Adds expectation for deleted entities #317
Merged
+125
−3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
||
| # mock `pandas.read_csv` to return the mock DataFrame | ||
| mock_df = pd.DataFrame({"resource": ["2f7d900dd48fd02"]}) | ||
| mocker.patch("pandas.read_csv", return_value=mock_df) |
There was a problem hiding this comment.
Are these lines necessary? I can't see where 'mock_df' is required.
There was a problem hiding this comment.
The setup mocks the read_csv() function, which is used to fetch resource data from the digital-land database.
Since pd.read_csv(base_url) returns a DataFrame, the mock_df is used to replicate that behaviour.
| # get dataset specific active resource list | ||
| params = urllib.parse.urlencode( | ||
| { | ||
| "sql": f"""select *,o.entity from reporting_historic_endpoints rhe join organisation o on rhe.organisation=o.organisation |
There was a problem hiding this comment.
For 'select *,o.entity ...' should there be an alias in front of the star or is 'o.entity' not required as the star will bring all of the columns in?
| test_fact_resource_data.to_sql( | ||
| "fact_resource", conn, if_exists="append", index=False | ||
| ) | ||
| test_fact_data.to_sql("fact", conn, if_exists="append", index=False) |
There was a problem hiding this comment.
Would we want the if_exists to be 'replace' as the tables are only meant for testing purposes it might append extra rows to a test table?
alexglasertpx
approved these changes
Jan 21, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this? (check all applicable)
Description
This PR creates an expectation to identify and return deleted entities for datasets.
The implementation follows these steps:
Fetch Active Resources: Retrieve the list of active resources for a given organisation.
Identify Active Entities: Gather the active entities associated with each resource.
Compare Entities: Compare the entities from the platform with those on the active resource.
If an entity exists on the platform but not in the active resource, it is flagged as deleted.
The entity numbers identified as deleted are returned in the results.
Related Tickets & Documents
QA Instructions, Screenshots, Recordings
Please replace this line with instructions on how to test your changes, a note
on the devices and browsers this has been tested on, as well as any relevant
images for UI changes.
Added/updated tests?
We encourage you to keep the code coverage percentage at 80% and above. Please refer to the Digital Land Testing Guidance for more information.
have not been included
[optional] Are there any post deployment tasks we need to perform?
[optional] Are there any dependencies on other PRs or Work?