Skip to content

Commit 3f2bacb

Browse files
committed
1. Merge branch 'dev' into leonlu2/time_values_and_issues
2 parents 2a13311 + 7643299 commit 3f2bacb

36 files changed

+430
-475
lines changed

docs/api/covidcast-signals/covid-act-now.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: COVID Act Now
3-
parent: Data Sources and Signals
3+
parent: Inactive Signals
44
grand_parent: COVIDcast Main Endpoint
55
---
66

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
---
2+
title: Quidel
3+
parent: Inactive Signals
4+
grand_parent: COVIDcast Main Endpoint
5+
---
6+
7+
# Quidel
8+
{: .no_toc}
9+
10+
* **Source name:** `quidel`
11+
12+
## Table of Contents
13+
{: .no_toc .text-delta}
14+
15+
1. TOC
16+
{:toc}
17+
18+
19+
## COVID-19 Tests
20+
These signals are still active. Documentation is available on the [Quidel page](quidel.md).
21+
22+
## Flu Tests
23+
24+
* **Earliest issue available:** April 29, 2020
25+
* **Last issued:** May 19, 2020
26+
* **Number of data revisions since May 19, 2020:** 0
27+
* **Date of last change:** Never
28+
* **Available for:** msa, state (see [geography coding docs](../covidcast_geography.md))
29+
* **Time type:** day (see [date format docs](../covidcast_times.md))
30+
31+
Data source based on flu lab tests, provided to us by Quidel, Inc. When a
32+
patient (whether at a doctor’s office, clinic, or hospital) has COVID-like
33+
symptoms, doctors may perform a flu test to rule out seasonal flu (influenza),
34+
because these two diseases have similar symptoms. Using this lab test data, we
35+
estimate the total number of flu tests per medical device (a measure of testing
36+
frequency), and the percentage of flu tests that are *negative* (since ruling
37+
out flu leaves open another cause---possibly covid---for the patient's
38+
symptoms), in a given location, on a given day.
39+
40+
The number of flu tests conducted in individual counties can be quite small, so
41+
we do not report these signals at the county level.
42+
43+
The flu test data is no longer updated as of May 19, 2020, as the number of flu
44+
tests conducted during the summer (outside of the normal flu season) is quite
45+
small. The data may be updated again when the Winter 2020 flu season begins.
46+
47+
| Signal | Description |
48+
| --- | --- |
49+
| `raw_pct_negative` | The percentage of flu tests that are negative, suggesting the patient's illness has another cause, possibly COVID-19 <br/> **Earliest date available:** 2020-01-31 |
50+
| `smoothed_pct_negative` | Same as above, but smoothed in time <br/> **Earliest date available:** 2020-01-31 |
51+
| `raw_tests_per_device` | The average number of flu tests conducted by each testing device; measures volume of testing <br/> **Earliest date available:** 2020-01-31 |
52+
| `smoothed_tests_per_device` | Same as above, but smoothed in time <br/> **Earliest date available:** 2020-01-31 |

docs/api/covidcast-signals/quidel.md

Lines changed: 2 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -157,34 +157,5 @@ not enough samples can be filled in from the parent state for smoothed signals s
157157
no data is reported for that area on that day; an API query for all reported geographic areas on
158158
that day will not include it.
159159

160-
## Flu Tests
161-
162-
* **Earliest issue available:** April 29, 2020
163-
* **Last issued:** May 19, 2020
164-
* **Number of data revisions since May 19, 2020:** 0
165-
* **Date of last change:** Never
166-
* **Available for:** msa, state (see [geography coding docs](../covidcast_geography.md))
167-
* **Time type:** day (see [date format docs](../covidcast_times.md))
168-
169-
Data source based on flu lab tests, provided to us by Quidel, Inc. When a
170-
patient (whether at a doctor’s office, clinic, or hospital) has COVID-like
171-
symptoms, doctors may perform a flu test to rule out seasonal flu (influenza),
172-
because these two diseases have similar symptoms. Using this lab test data, we
173-
estimate the total number of flu tests per medical device (a measure of testing
174-
frequency), and the percentage of flu tests that are *negative* (since ruling
175-
out flu leaves open another cause---possibly covid---for the patient's
176-
symptoms), in a given location, on a given day.
177-
178-
The number of flu tests conducted in individual counties can be quite small, so
179-
we do not report these signals at the county level.
180-
181-
The flu test data is no longer updated as of May 19, 2020, as the number of flu
182-
tests conducted during the summer (outside of the normal flu season) is quite
183-
small. The data may be updated again when the Winter 2020 flu season begins.
184-
185-
| Signal | Description |
186-
| --- | --- |
187-
| `raw_pct_negative` | The percentage of flu tests that are negative, suggesting the patient's illness has another cause, possibly COVID-19 <br/> **Earliest date available:** 2020-01-31 |
188-
| `smoothed_pct_negative` | Same as above, but smoothed in time <br/> **Earliest date available:** 2020-01-31 |
189-
| `raw_tests_per_device` | The average number of flu tests conducted by each testing device; measures volume of testing <br/> **Earliest date available:** 2020-01-31 |
190-
| `smoothed_tests_per_device` | Same as above, but smoothed in time <br/> **Earliest date available:** 2020-01-31 |
160+
## Flu Tests (inactive)
161+
These signals were updated until May 19, 2020. Documentation is still available on the [inactive Quidel page](quidel-inactive.md).

docs/api/covidcast-signals/safegraph.md

Lines changed: 0 additions & 87 deletions
This file was deleted.

docs/api/covidcast-signals/usa-facts.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: USAFacts Cases and Deaths
3-
parent: Data Sources and Signals
3+
parent: Inactive Signals
44
grand_parent: COVIDcast Main Endpoint
55
---
66

src/server/_params.py

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99
from ._exceptions import ValidationFailedException
1010
from .utils import days_in_range, weeks_in_range, guess_time_value_is_day, guess_time_value_is_week, IntRange, TimeValues, days_to_ranges, weeks_to_ranges
11+
from ._validate import require_any, require_all
1112

1213

1314
def _parse_common_multi_arg(key: str) -> List[Tuple[str, Union[bool, Sequence[str]]]]:
@@ -308,3 +309,177 @@ def parse_day_or_week_range_arg(key: str) -> TimeSet:
308309
if is_week:
309310
return TimeSet("week", [parse_week_range_arg(key)])
310311
return TimeSet("day", [parse_day_range_arg(key)])
312+
313+
314+
def _extract_value(key: Union[str, Sequence[str]]) -> Optional[str]:
315+
if isinstance(key, str):
316+
return request.values.get(key)
317+
for k in key:
318+
if k in request.values:
319+
return request.values[k]
320+
return None
321+
322+
323+
def _extract_list_value(key: Union[str, Sequence[str]]) -> List[str]:
324+
if isinstance(key, str):
325+
return request.values.getlist(key)
326+
for k in key:
327+
if k in request.values:
328+
return request.values.getlist(k)
329+
return []
330+
331+
332+
def extract_strings(key: Union[str, Sequence[str]]) -> Optional[List[str]]:
333+
s = _extract_list_value(key)
334+
if not s:
335+
# nothing to do
336+
return None
337+
# we can have multiple values
338+
return [v for vs in s for v in vs.split(",")]
339+
340+
341+
def extract_integer(key: Union[str, Sequence[str]]) -> Optional[int]:
342+
s = _extract_value(key)
343+
if not s:
344+
# nothing to do
345+
return None
346+
try:
347+
return int(s)
348+
except ValueError:
349+
raise ValidationFailedException(f"{key}: not a number: {s}")
350+
351+
352+
def extract_integers(key: Union[str, Sequence[str]]) -> Optional[List[IntRange]]:
353+
parts = extract_strings(key)
354+
if not parts:
355+
# nothing to do
356+
return None
357+
358+
def _parse_range(part: str):
359+
if "-" not in part:
360+
return int(part)
361+
r = part.split("-", 2)
362+
first = int(r[0])
363+
last = int(r[1])
364+
if first == last:
365+
# the first and last numbers are the same, just treat it as a singe value
366+
return first
367+
elif last > first:
368+
# add the range as an array
369+
return (first, last)
370+
# the range is inverted, this is an error
371+
raise ValidationFailedException(f"{key}: the given range is inverted")
372+
373+
try:
374+
values = [_parse_range(part) for part in parts]
375+
# check for invalid values
376+
return None if any(v is None for v in values) else values
377+
except ValueError as e:
378+
raise ValidationFailedException(f"{key}: not a number: {str(e)}")
379+
380+
381+
def parse_date(s: str) -> int:
382+
# parses a given string in format YYYYMMDD or YYYY-MM-DD to a number in the form YYYYMMDD
383+
try:
384+
return int(s.replace("-", ""))
385+
except ValueError:
386+
raise ValidationFailedException(f"not a valid date: {s}")
387+
388+
389+
def extract_date(key: Union[str, Sequence[str]]) -> Optional[int]:
390+
s = _extract_value(key)
391+
if not s:
392+
return None
393+
return parse_date(s)
394+
395+
396+
def extract_dates(key: Union[str, Sequence[str]]) -> Optional[TimeValues]:
397+
parts = extract_strings(key)
398+
if not parts:
399+
return None
400+
values: TimeValues = []
401+
402+
def push_range(first: str, last: str):
403+
first_d = parse_date(first)
404+
last_d = parse_date(last)
405+
if first_d == last_d:
406+
# the first and last numbers are the same, just treat it as a singe value
407+
return first_d
408+
if last_d > first_d:
409+
# add the range as an array
410+
return (first_d, last_d)
411+
# the range is inverted, this is an error
412+
raise ValidationFailedException(f"{key}: the given range is inverted")
413+
414+
for part in parts:
415+
if "-" not in part and ":" not in part:
416+
# YYYYMMDD
417+
values.append(parse_date(part))
418+
continue
419+
if ":" in part:
420+
# YYYY-MM-DD:YYYY-MM-DD
421+
range_part = part.split(":", 2)
422+
r = push_range(range_part[0], range_part[1])
423+
if r is None:
424+
return None
425+
values.append(r)
426+
continue
427+
# YYYY-MM-DD or YYYYMMDD-YYYYMMDD
428+
# split on the dash
429+
range_part = part.split("-")
430+
if len(range_part) == 2:
431+
# YYYYMMDD-YYYYMMDD
432+
r = push_range(range_part[0], range_part[1])
433+
if r is None:
434+
return None
435+
values.append(r)
436+
continue
437+
# YYYY-MM-DD
438+
values.append(parse_date(part))
439+
# success, return the list
440+
return values
441+
442+
def parse_source_signal_sets() -> List[SourceSignalSet]:
443+
ds = request.values.get("data_source")
444+
if ds:
445+
# old version
446+
require_any("signal", "signals", empty=True)
447+
signals = extract_strings(("signals", "signal"))
448+
if len(signals) == 1 and signals[0] == "*":
449+
return [SourceSignalSet(ds, True)]
450+
return [SourceSignalSet(ds, signals)]
451+
452+
if ":" not in request.values.get("signal", ""):
453+
raise ValidationFailedException("missing parameter: signal or (data_source and signal[s])")
454+
455+
return parse_source_signal_arg()
456+
457+
458+
def parse_geo_sets() -> List[GeoSet]:
459+
geo_type = request.values.get("geo_type")
460+
if geo_type:
461+
# old version
462+
require_any("geo_value", "geo_values", empty=True)
463+
geo_values = extract_strings(("geo_values", "geo_value"))
464+
if len(geo_values) == 1 and geo_values[0] == "*":
465+
return [GeoSet(geo_type, True)]
466+
return [GeoSet(geo_type, geo_values)]
467+
468+
if ":" not in request.values.get("geo", ""):
469+
raise ValidationFailedException("missing parameter: geo or (geo_type and geo_value[s])")
470+
471+
return parse_geo_arg()
472+
473+
474+
def parse_time_set() -> TimeSet:
475+
time_type = request.values.get("time_type")
476+
if time_type:
477+
# old version
478+
require_all("time_type", "time_values")
479+
time_values = extract_dates("time_values")
480+
return TimeSet(time_type, time_values)
481+
482+
if ":" not in request.values.get("time", ""):
483+
raise ValidationFailedException("missing parameter: time or (time_type and time_values)")
484+
485+
return parse_time_arg()

src/server/_query.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,7 @@
1818
from ._common import db
1919
from ._printer import create_printer, APrinter
2020
from ._exceptions import DatabaseErrorException
21-
from ._validate import extract_strings
22-
from ._params import GeoSet, SourceSignalSet, TimeSet
21+
from ._params import extract_strings, GeoSet, SourceSignalSet, TimeSet
2322
from .utils import time_values_to_ranges, IntRange, TimeValues
2423

2524

0 commit comments

Comments
 (0)