Migrate AWS to Archive

[Makosak]

Migrate AWS account to UIUC AWS for long-term archiving and storage.

[mradamcox]

I believe the final step for this is the geoda-covid-atlas bucket on the old AWS account. This needs to be copied to the new account and any references to it in the code need to be updated. Also, there is a CloudFront distribution on top of this bucket. It will warrant a little investigation into whether this is really a necessary part of the infrastructure, i.e. how much is it really helping vs just direct requests to S3.

Once the bucket is transferred, check the following for references to the bucket name or distribution ID, and update as needed:

• workflow env vars
• netlify deploy env vars
• hard-coded links within the code base

[mradamcox]

The following steps were taken to complete this:

• Create bucket geoda-covid-atlas2 in new AWS account with the following policy to allow public reading of the data (same policy as the original bucket had):

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Sid": "CDN-Allow",
              "Effect": "Allow",
              "Principal": "*",
              "Action": "s3:GetObject",
              "Resource": "arn:aws:s3:::geoda-covid-atlas2/*"
          }
      ]
  }
  

• Sync the original bucket to this new bucket:
  aws s3 sync s3://geoda-covid-atlas s3://geoda-covid-atlas2
  • Note: to complete this I had to add s3:GetObjectTagging permissions to the original bucket policy
• Find all references in the code base to geoda-covid-atlas as bucket name and change to geoda-covid-atlas2 (these were all in py files)
• Find reference in the code base to the CloudFront distribution and change to the base url for the S3 bucket
  • The CDN was only used to download 3 zip files in the fetchLatestData script so I don't think that's enough usage to warrant this extra level of infrastructure.
• I made a ticket on the archive repo AWS credentials and S3 bucket must be updated for future use covid#304 about how that code base would need to be updated to use the migrated content, if it were ever reactivated.
• Created a new user in AWS with CLI access to the geoda-covid-atlas2 bucket, and updated the github actions environment variables in this repo with these credentials.
• Checked Netlify and confirmed there are no env vars there related to AWS resources, besides REACT_APP_STORIES_PUBLIC_URL which had already been updated.