data can be missed

[lila]

With the current approach -- with daily ingestion of data from the previous day -- data may be missed if the data has not been synced to the fitbit backend when the processes pull the data. One problem is that in general, the data may be missing

• because it does not exist, for example, if the battery has died no data would be generated.
• or the data may have been generated, but until the mobile phone app syncs the data, it won't show up on the backend web apis.

Similarly, if data is ingested multiple times, does the data get overwritten, or appended only?

[lila]

the ingestion routes do offer options for specifying date and user parameters. if data is specified, the data from date is ingested instead of yesterday. likewise, if user is specified, then only that user's data is pulled. for example:

http://<data connector url>/fitbit_activity_scope&[email protected]&date=2022-09-01 would only pull data for that one user for that one day. the format for the date is: yyyy-MM-dd.

by default the urls are only reachable with auth headers, and should not be publicly accessible. but you can hit that url from the commandline like this:

curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" SERVICE_URL

assuming that the user has access to the service url.