Merge pull request #166 from cmu-delphi/release_v4.0

Release v4.0

Author: kateharwood

.github/workflows/s3_upload_ec2.yml

@@ -19,6 +19,9 @@ jobs:
 
     # Steps represent a sequence of tasks that will be executed as part of the job
     steps:
+      - name: Change dir owner to working user
+        run: sudo chown -R $USER:$USER $GITHUB_WORKSPACE
+
       # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
       - uses: actions/checkout@v2
 

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: forecast-eval
 Title: Forecast Evaluation Dashboard
-Version: 3.1
+Version: 4.0
 Authors@R: person("Kate", "Harwood",
                   role = c("cre")), 
 	   person("Chris", "Scott",

README.md

@@ -1,12 +1,24 @@
 # Forecast Eval
 
 Epiforecast scoring and interactive evaluation tools.
+https://delphi.cmu.edu/forecast-eval/
+
+Code on the `dev` branch appears in staging: https://staging.delphi.cmu.edu/forecast-eval/  
+Username and password are included in the [meeting notes doc](https://docs.google.com/document/d/1q8sKrbjzymEDsWQ9mUomOZ255-_5W6RPGgTdFlHmpmE/edit)
 
 ## Steps to contribute
 1. Create a new branch off of `dev`
 2. Create a pull request into `dev`
 
-Branch `main` is the production branch. Branch `dev` will be merged into main when a release is ready.
+Branch `main` is the production branch. Branch `dev` will be merged into main when a release is ready. See below for instructions on how to create a release.
+
+
+
+**Note:** the easiest way to view and develop this project locally is to use RStudio and run the RShiny app from inside the IDE
+
+<img width="1111" alt="Screen Shot 2021-08-30 at 10 56 59 AM" src="https://user-images.githubusercontent.com/14190352/131359925-3b460d21-b9aa-4a40-a691-cd705ab98431.png">
+
+Alternatively, ...
 
 
 ## Building
@@ -52,6 +64,21 @@ git push origin v1.0
 Create a PR into `main`.  
 After code is merged to `main`, perform cleanup by merging `main` into `dev` so that `dev` stays up to date.
 
+# Code Structure
+ - `workflows` contains the weekly data pipeline workflow action (`s3_upload_ec2.yml`) and the `main.yml` that runs on branch merge
+ - `Report` contains the scoring and data upload scripts that run weekly
+ - `dashboard` contains all the code for the RShiny dashboard
+   - `www` contains the styling and the assets
+   - `app.R` is the main RShiny file with the UI and server functions
+   - `common.R` is for code shared between the app and the download feature
+   - `export_scores.R` contains the code for the download feature
+   - `about.md` contains the code for the "About" tab in the dasboard (other .md files contain explanations of the scores and other text info that appears in the app)
+ - `docker_buid` contains the `Dockerfile` specifying the version of the `covidcast` docker image to use
+ - `docker_dashboard` contains the `Dockerfile` and `shiny_server.conf` for the RShiny app
+   - ***Note: when adding a new package dependency to the app, it must be specified in this Dockerfile***
+ - `DESCRIPTION` is where the version number is updated for each release
+ - `Makefile` contains all commands to build and run the dashboard and score and upload the data
+
 ## Note on Scoring Script
 
 When updates are made in the evalcast package that affect the scoring script, the covidcast docker image must be rebuilt by kicking off the workflow here: https://github.com/cmu-delphi/covidcast-docker/actions/workflows/main.yml. Ensure that the changes in evalcast will be compatible with the dashboard and will not cause errors - when the scoring script is run on these changes the results show up automatically in prod.

dashboard/about.md

@@ -79,9 +79,11 @@ Though hospitalizations are forecasted on a daily basis, in keeping with the cas
 #### **Notes on the Data**
 
 *   If a forecast does not include an explicit point estimate, the 0.5 quantile is taken as the point estimate for calculating absolute error.
-*   WIS is only shown for forecasts that have predictions for all quantiles (23 quantiles for deaths and hospitalizations and 7 for cases)
+*   The weighted interval score is only shown for forecasts that have predictions for all quantiles (23 quantiles for deaths and hospitalizations and 7 for cases)
 *   Totaling over all states and territories does not include nationwide forecasts. To ensure that values are comparable, these totals also exclude any locations that are absent from any file that was submitted by one of the selected forecasters.
-*   We include revisions of observed values, which means that the scores for forecasts made in the past can change as our understanding of the ground truth changes.
+*   For scoring, we include revisions of observed values, which means that the scores for forecasts made in the past can change as our understanding of the ground truth changes.
+*   The observed data can also be viewed **'as of'** a certain date, which shows what observed data a forecaster had available
+when a past forecast was made (but the forecasts are always scored on the latest revision of the observed data).
 
 #### **Accessing the Data**
 The forecasts and scores are available as RDS files and are uploaded weekly to a publicly accessible AWS bucket.