-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers
Description
Using RDS to read/write/store data is slow and not portable. We may want to switch the data pipeline or dashboard to being written in Python in the future, so using a format that Python can easily read would be preferable.
Looking at a comparison of different formats and packages, storing as a CSV and reading/writing using data.table
via fread
and fwrite
seems pretty good. feather
could also be an option (is supported in both R and Python) but it's not clear if standard dataframe and dplyr
procedures work with it. data.table
data can be seamlessly processed either with dplyr
or with faster data.table
syntax.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers