Skip to content

850 avoid search by id before every server mode download #916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 8 commits into from
Closed

850 avoid search by id before every server mode download #916

wants to merge 8 commits into from

Conversation

@jlahovnik
Copy link
Collaborator

@jlahovnik jlahovnik commented Nov 8, 2023 •
edited
Loading

  • download info is stored in instance of EODataAccessGateway with key composed of product type, provider and id
  • if download is attempted for product where id is not in download info and search by id is available for provider and product type, it will be executed; if search by id is not possible an error will be raised to do a search

@jlahovnik jlahovnik linked an issue Nov 8, 2023 that may be closed by this pull request
avoid search by id before every server mode download #850
Closed
@jlahovnik jlahovnik self-assigned this Nov 8, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Nov 8, 2023 •
edited
Loading

Test Results

    4 files  ±0      4 suites  ±0   4m 3s ⏱️ ±0s
  442 tests +1    439 ✅ +1   3 💤 ±0  0 ❌ ±0 
1 768 runs  +4  1 688 ✅ +2  80 💤 +2  0 ❌ ±0 

Results for commit 49974c0. ± Comparison against base commit 9b95224.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 8, 2023 •
edited
Loading

Code Coverage (Ubuntu)

File Coverage
All files 82% ✅

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against 49974c0

@github-actions
Copy link
Contributor

github-actions bot commented Nov 8, 2023 •
edited
Loading

Code Coverage (Windows)

File Coverage
All files 76% ✅

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against 49974c0

@alambare alambare marked this pull request as draft November 16, 2023 16:36
@jlahovnik jlahovnik force-pushed the 850-avoid-search-by-id-before-every-server-mode-download branch 2 times, most recently from 83d5e2c to 0e981c2 Compare December 6, 2023 12:42
@jlahovnik
Copy link
Collaborator Author

jlahovnik commented Dec 6, 2023

updated based on input from @alambare-csgroup

@jlahovnik jlahovnik marked this pull request as ready for review December 6, 2023 15:46
sbrunato
sbrunato requested changes Dec 21, 2023
View reviewed changes
eodag/api/core.py
os.path.join(self.conf_dir, "shp"),
)
self.set_locations_conf(locations_conf_path)
self.download_info = {}
Copy link
Collaborator

@sbrunato sbrunato Dec 21, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please store full SearchResult objects in a _search_results_cache collections.dequeue() having maxlen default to 100 (stored in a new constant) and customizable using an environment variable like EODAG_SEARCH_RESULTS_CACHE_MAXLEN.

Please also put _search_results_cache in rest.utils as it should only concern server-mode downloads.

To find a matching product, loop over the cache starting with the most recent result. (check at first only provider and product type of the first product of each search result)

@sbrunato sbrunato marked this pull request as draft December 21, 2023 08:01
@jlahovnik jlahovnik force-pushed the 850-avoid-search-by-id-before-every-server-mode-download branch from c771dec to f59180a Compare January 4, 2024 08:40
@jlahovnik jlahovnik closed this Feb 6, 2024
@jlahovnik
Copy link
Collaborator Author

jlahovnik commented Feb 6, 2024

closed because change requests were implemented on new branch -> new PR: #1012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sbrunato sbrunato sbrunato requested changes

Assignees

@jlahovnik jlahovnik

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

avoid search by id before every server mode download

3 participants

@jlahovnik @sbrunato