Skip to content

Add dummy dampening to non-stationary models to allow stationary initialization #486

Open
@jessegrabowski

Description

@jessegrabowski

This is a trick I did in the ETS model. I think we should use it everywhere, because asking users to set priors on P0 is the most obnoxious part of the statespace package right now.

The context is that the Kalman filter iteratively estimates the covariance matrix of residuals between the observed data and the given time series model. This matrix is called P. It has to start somewhere, but the starting position really doesn't matter much -- the Kalman filter will converge to a fixed point after a few dozen iterations. See this issue for some commentary about convergence speed.

So P0 is basically a nuisance parameter. The data is very weakly informative about it (see here) because of how short lasting its influence is. We just basically need to put something sensible there. Actually you can also put absolute nonsense there -- statsmodels, for instance, just puts 1e9. I don't want to do that, because it makes the HDIs of the first few obsevations look horrible.

When a model is stationary, there's actually a closed form initialization for P0. This is already used in VARMA and SARIMA models (see here, for example). But models with a level component, like ETS and LevelTrend, are not stationary, and cannot be initialized this way.

What I did in ETS was to give an option to pretend as if the model was stationary. Consider a level-trend transition matrix, which looks like:

$$ T = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} $$

This is not stationary, because it has eigenvalues that are 1 or greater (specifically, both eigenvalues are 1). But we can pretend it's stationary over a long time horizon, by adding a dampening term $\rho \approx 1$. So then:

$$ T = \begin{bmatrix} \rho & 1 \\ 0 & \rho \end{bmatrix} $$

The eigenvalues of this matrix will be $[ \rho, \rho ]$, so we can choose something like 0.99 and then do stationary initialization. After initialization, we rewrite the graph to replace rho with constant 1. This dampened system will be quite similar to the original system, and should result in sane initialization without requiring the user to do anything. We can allow the user to choose the value of rho, or to turn off this scheme if she's a real control freak.

Basically this issue asks for this scheme to be applied to all non-stationary components (seasonality and trend, basically).

As part of this, we might also want to have each component provide it's own P0, which can be pre-initialized using the stationarity scheme. If not, the model should ask for P0 block-by-block, rather than asking for one large matrix. It's always valid to block initialize P0, and doing a bunch of small initializations (some of which have easy stationary defaults) will be easier on users.

We could add an option to build to throw all these blockwise P0s away and replace it with a single large matrix (the current behavior) in case someone wants that.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions