Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an endpoint to signal_dashboard that returns report freshness #565

Open
krivard opened this issue Jun 7, 2021 · 3 comments
Open

Add an endpoint to signal_dashboard that returns report freshness #565

krivard opened this issue Jun 7, 2021 · 3 comments
Labels
api change affect the API and its responses enhancement

Comments

@krivard
Copy link
Contributor

krivard commented Jun 7, 2021

From RoniRos in a thread from the indicator status/backfill naming doc:

What I have in mind for 'historical availability' is something much simpler than the (admittedly very complex) notion of "what was known when". I just want to know, for each signal, how "fresh" it has been, for some simplistic definition of "fresh". Here is one definition: for a given signal and a given reporting date, report the smallest lag available for that date. ~That's 40 integers per day. Later on, if we want to get fancy, we can make that specific to each location.

(crossouts are mine; we actually have ~150 unique source+signal pairs)

For v1, rather than caching this information we should just compute it at query time.

Easiest & quickest implementation I can think of is the following:

Given: reporting_date, source, signal
Compute:

min(lag) 
WHERE issue=reporting_date 
AND source=source 
AND signal=signal

A slightly more robust (in case of reporting gaps), but slower-to-compute implementation is:

Given: reporting_date, source, signal
Compute:

min(lag) 
WHERE issue<=reporting_date 
AND source=source 
AND signal=signal 
AND time_value=(
    max(time_value) where issue<=reporting_date` AND source=source AND signal=signal
)

┆Issue is synchronized with this Asana task by Unito

@RoniRos
Copy link
Member

RoniRos commented Jun 7, 2021

Thanks.
150 signals. Oh Mine. I presume the majority are survey marginals?

@RoniRos
Copy link
Member

RoniRos commented Jun 7, 2021

Just to clarify: once this endpoint is added, we will add a way to visualize this summary statistics for all (desired) signals in the dashboard - correct?

@krivard
Copy link
Contributor Author

krivard commented Jun 7, 2021

once this is added, we'll track visualization in a separate task

@sgratzl sgratzl added api change affect the API and its responses enhancement labels Jun 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api change affect the API and its responses enhancement
Projects
None yet
Development

No branches or pull requests

3 participants