Overview

This repository contains the data engineering code for the Nonprofit Trends and Impacts survey dashboard.

Requirements

Always check that the latest template is downloaded and specified in R/config.R

• Y:// drive access
• Google Sheets access to the metricsMaster and dataTemplate sheet.
• Alternative data template URL. Has download links for dataTemplate and metricsmaster.

Directory

• R contains the data engineering scrips in the order they should be run. Non-numbered scripts containing either helper functions or configuration parameters.
• data contains raw and processed data, and datasets used for validation.
  • raw: stores raw data downloaded from Y drive or Google Sheets, for validation datasets
  • intermediate: Intermediate datasets
  • processed: Final datasets _filtered: dataset for dashboard _formatted: string formatting without removing metrics where n < 25 _full_postprocessed: numeric variables for validation/inspection
  • validate
    • Validation datasets used in unittests.
• data_requests contains additional code used to run ad-hoc data pulls.
• tests contains testthat unit tests