Create forecasting interface

[damonbayer]

Goal

We should have a consistent interface for creating forecasts from models.

Context

For implementation inspiration, we can look at the NumPyro Time Series Forecasting example.

[SamuelBrand1]

Following from f2f discussion, its worth considering this problem: CDCgov/Rt-without-renewal#243

Underlying difficulty is using Predictive, you should check if it works on an AR process like here https://num.pyro.ai/en/latest/examples/ar2.html because its a different inference process (stochastic even when conditioned on parameters).

[damonbayer]

There is some discussion of forecasting for the AR2 NumPyro demo here: pyro-ppl/numpyro#1350

[damonbayer]

See also https://juanitorduz.github.io/arma_numpyro/

[SamuelBrand1]

See also https://juanitorduz.github.io/arma_numpyro/

This is nice example... but looks a bit weird?

I think statsmodel use MLE inference under the hood? In which case I'm not sure I'd expect the forecast to look so similar to one fitted with NUTS?

[damonbayer]

@SamuelBrand1 weird in what way?

[SamuelBrand1]

That it looks so similar to statsmodel given its a different inference method? It doesn't look like the posteriors are super peaked?

[SamuelBrand1]

Looking at the code, the predictive mode switch looks like it makes sense.

[damonbayer]

Does anyone have a forecasting interface (i.e. the function signature for creating a forecast) that they really like that I might use for inspiration? Could be in Python, R, Julia, etc. @SamuelBrand1 @seabbs @dylanhmorris

[damonbayer]

This discussion on the numpyro forum looks promising.

Also noting that there is a GaussianRandomWalk class built in to numpyro that we could take inspiration from.

[SamuelBrand1]

This discussion on the numpyro forum looks promising.

Yeah, this example points at what I was worried about with forecasting (i.e. when you have a stochastic distribution in the future as opposed to the time series example with numpyro docs).

From my basic looking around it seems that the most common solution in numpyro land is to have some kind of conditional statement inside the def of the sampleable model... that seems to do the job but maybe seems like potentially a lot of boiler plate? E.g. would we get pushed towards needing to write an extra conditional chunk of all the models at the top level that we want to forecast from?

[SamuelBrand1]

Also noting that there is a GaussianRandomWalk class built in to numpyro that we could take inspiration from.

Is there a downside to just importing it? If its been provided by numpyro devs then it might be better in some way that what we do ourselves?

[SamuelBrand1]

Does anyone have a forecasting interface (i.e. the function signature for creating a forecast) that they really like that I might use for inspiration? Could be in Python, R, Julia, etc. @SamuelBrand1 @seabbs @dylanhmorris

I like the epidemia interface for prediction/counterfactual: https://imperialcollegelondon.github.io/epidemia/reference/posterior_predict.epimodel.html .

The tl; dr explanation is they fit an object (which contains posterior draws for the params) and then passing newdata can create forecasts (by being longer) and/or counterfactuals (by having different covariates like an earlier lockdown). Its demoed here, https://imperialcollegelondon.github.io/epidemia/articles/europe-covid.html .