Open
Description
While looking into memory and runtime of the predictions cards script, I noticed that we pull a lot more data incrementally than I would've expected:
[1] "Out of 2026 previous forecasts we will pull 664 new forecasts\n"
1/78:Auquan-SEIR with 12 forecasts ...
2/78:CovidAnalytics-DELPHI with 11 forecasts ...
3/78:UMass-MechBayes with 2 forecasts ...
4/78:CMU-TimeSeries with 0 forecasts ...
5/78:Columbia_UNC-SurvCon with 1 forecasts ...
6/78:Geneva-DetGrowth with 19 forecasts ...
7/78:MIT_CritData-GBCF with 2 forecasts ...
8/78:UA-EpiCovDA with 5 forecasts ...
9/78:UCSB-ACTS with 0 forecasts ...
10/78:GT_CHHS-COVID19 with 7 forecasts ...
11/78:Imperial-ensemble2 with 0 forecasts ...
12/78:JHUAPL-Bucky with 0 forecasts ...
13/78:UCSD_NEU-DeepGLEAM with 0 forecasts ...
14/78:Caltech-CS156 with 0 forecasts ...
15/78:UCF-AEM with 0 forecasts ...
16/78:FAIR-NRAR with 82 forecasts ...
I suspect that these are all forecasts that are filtered out due to our inclusion criteria after everything has been pulled down. We should consider tracking forecasts separately on if they were included. Not only would we be able to surface more detail on specific forecasters on the front-end, but we wouldn't have to pull down previously filtered forecasts every time the prediction cards are updated.