Understanding State-Space Models (An Inflation Example)

by Eric · Published August 3, 2022 · Updated August 10, 2022

Introduction

State-space models provide a powerful environment for modeling dynamic systems. Their flexibility has resulted in a wide variety of applications across fields including radar tracking, 3-D modeling, monetary policy modeling, weather forecasting, and more.

In this blog, we look more closely at state-space modeling using a simple time series model of inflation.

We will cover:

• The components of state-space models.
• Representing state-space models in GAUSS.
• Estimating model parameters using state-space models.

Inflation Modeling

Today we will use the state-space framework to build a simple, albeit naive, model of inflation. This model uses a CPI inflation index, created from the FRED CPIAUCNS average quarterly dataset. Our data ranges from 1971q1 to 2021q4.

State-Space Model Specification

State-space systems specify the dynamics and relationship of two components:

• Observed data.
• Unobserved state variable.

These components are modeled using a measurement equation for the observed data,

$$ y_t = d + Z\alpha_t + \epsilon_t $$

and a transition equation for the unobserved state variable,

$$ \alpha_{t+1} = c + T\alpha_t + R\eta_t $$

where

$$ \epsilon_t \sim N(0, H) $$ $$ \eta_t \sim N(0, Q) $$

and

While this representation may seem simplistic, the framework offers a very flexible platform for modeling. It is used widely in time series analysis including ARIMA, SARIMA, vector autoregressive models, unobserved components, and dynamic factor models.

Example Representation: AR(2) Model

As an example consider an $AR(2)$ model such that

$$ y_t = \phi_1 y_{t-1} + \phi_2 y_{t-2} + e_t $$
$$ e_t \sim N(0, \sigma^2) $$

Let

$$ \alpha_t = (y_t, y_{t-1})' $$

Then we can write our $AR(2)$ model such that

where

Example Representation: ARMA(1,1)

The $ARMA(1,1)$ model is given by

$$ y_t = \phi y_{t-1} + \epsilon_t + \theta \epsilon_{t-1} $$ $$ \epsilon_t \sim N(0, \sigma^2) $$

If we let $ \alpha_t = (y_t, \theta \epsilon_t)$ then this can be represented in state-space form such that

where

Estimation of State-Space Models

In most cases, the parameters of our state-space representation are unknown and we wish to estimate them. However, this requires not only estimating the parameters but also estimating the unobserved state variable. Fortunately, we can do both using a combination of the Kalman filter and maximum likelihood estimation.

Let's look at estimating our $ARMA(1,1)$ model using the new sslib library.

The sslib Library

The sslib library is a GAUSS application module for setting up and estimating state-space models. The state-space library requires:

1. A working copy of GAUSS 22+.
2. The Constrained Maximum Likelihood MT library for GAUSS.
3. The Time Series MT library for GAUSS.

Loading and Transforming Data

To begin our state-space inflation models we need to:

• Load the required libraries:cmlmt, tsmt, and sslib.
• Load the CPI data.
• Calculate the inflation rate.

new;

/*
** Load required libraries
*/
library cmlmt, tsmt, sslib;

/*
** Perform import
*/
cpi_data = loadd(__FILE_DIR $+ "cpi_fred_q_ext.csv", "date($date) + cpi");

// Filter to exclude 2022 data
cpi_data = selif(cpi_data, cpi_data[., "date"] .< "2022");

/*
** Calculate log difference 
** annual inflation rate
*/

// Compute log difference rate
y = tsdiff(ln(cpi_data[., "cpi"]), 1);

// Convert from average quarterly decimal
// to annual percentage 
y = 4*y*100;

// Rename y variable
y = dfname(y, "Inflation", "cpi");

Setting Up the Parameter Vector and Start Values

Our inflation model has three unknown parameters that we will estimate, $\phi$, $\theta$, and $\sigma^2$. This means our initial parameter vector will be a $3 \times 1$ column vector of starting values.

/*
** Step two: Set up parameter vector 
**           and start values
*/
param_vec_st = asDF(zeros(3, 1), "param");

param_vec_st[1] = 0;
param_vec_st[2] = 0;
param_vec_st[3] = 1;

Initializing the Control Structure and System Matrices

The sslib uses a ssControl structure to:

1. Specify the state-space system matrices.
2. Implement stationarity and non-negativity constraints on parameters.
3. Control modeling features.
4. Specify advanced maximum likelihood controls.

The first step to initializing our model is specifying our model dimensions. These values will be used to initialize the system matrices.

The model dimensions to be specified include:

/*
** Step three: Set up control structure
**             and model matrices.
*/
// Number of endogenous variables
k_endog = 1;

// Number of states
k_states = 2;

Next we initialize our ssControl structure and set the default system matrices using ssControlCreate:

// Declare and instance of control structure
struct ssControl ssCtl;
ssCtl = ssControlCreate(k_states, k_endog);

// Specify parameter names for output printing
ssCtl.param_names = "phi"$|"theta"$|"sigma2";

Specifying the State-Space Representation

After initializing the state-space system matrices, we are ready to specify our state-space representation. This is done in two separate steps:

1. Specify the fixed system matrices that do not contain parameters.
2. Specify the matrices which do contain parameters using a custom updateSSModel function.

The system matrices are stored in the initialize ssControl in the ssModel structure named ssm.

Specifying Fixed System Matrices

To specify the fixed system matrices we use standard matrix assignment:

// Set fixed parameters of model
ssctl.ssm.Z = { 1  0 };
ssctl.ssm.H = 0;

Specifying System Matrices With Parameters

To specify the relationship between the model parameters and the state-space system matrices, we use an updateSSModel procedure. This procedure is then passed to the ssFit procedure for estimation.

The updateSSModel procedure should always contain two inputs:

Since this structure uses a pointer to the *ssmod structure, we use the arrow notation, ->, for assigning values to members within the structures.

 /*
** Set up procedure for updating SS model 
** structure.
**
*/
proc (0) = updateSSModel(struct ssModel *ssmod, param);

    // Set up kalman filter matrices
    ssmod->R = 1|param[2];
    ssmod->T = (param[1]~1)|(0~0);
    ssmod->Q = param[3];

endp;

Parameter constraints

The final step before estimation is to specify our parameter constraints. For our $ARMA(1,1)$ model we need to:

1. Constrain $\phi$ and $\theta$ to be stationary/invertible using the stationary_vars member of the ssControl structure.
2. Constrain $\sigma^2$ to be non-negative using the positive_vars member of the ssControl structure.

/*
** Constrained variables
*/

/* 
** This stationary_vars member
** indicates which variables should be 
** constrained to stationarity.
*/
// Set the first and second parameters in
// the parameter vector to be stationary
ssCtl.stationary_vars = 1|2;

/* 
** This positive_vars member
** indicates which variables should be 
** constrained to be positive.
*/
// Set the third parameter in
// the parameter vector to be positive
ssCtl.positive_vars = 3;

Estimation

Once the model is specified and the constraints are set, the parameters are estimated using the ssFit procedure. This procedure requires four inputs:

&updateSSModel

A pointer to a procedure that updates the state-space system matrices with the parameters.

param_vec_st

Vector, starting parameter values.

y

Vector, the response data.

ssCtl

Structure, an instance of the ssControl structure used to control features of the state-space model, Kalman filter, and maximum likelihood estimation.

/*
** Step six: Call the ssFit procedure.
**            This will:
**              1. Estimate model parameters.
**              2. Estimate inference statistics (se, t-stats).
**              3. Perform model residual diagnostics.
**              4. Compute model diagnostics and summary statistics.
*/
struct ssOut sOut;
sOut = ssFit(&updateSSModel, param_vec_st, y, ssCtl); 

The ssFit procedure stores results to an ssOut structure and prints estimates, inference statistics, and model diagnostics to screen:

Return Code:                                                             0
Log-likelihood:                                                     -492.4
Number of Cases:                                                       202
AIC:                                                                 990.7
AICC:                                                                990.8
BIC:                                                                  1001
HQIC:                                                                989.7
Covariance Method:                                    ML covariance matrix
==========================================================================

  Parameters   Estimates   Std. Err.      T-stat       Prob.    Gradient
--------------------------------------------------------------------------
         phi      0.9810      0.0160     61.2187      0.0000     -0.0026
       theta     -0.6266      0.0682     -9.1923      0.0000      0.0005
      sigma2      6.5535      0.6537     10.0246      0.0000      0.0000

Wald 95% Confidence Limits
--------------------------------------------------------------------------
  Parameters   Estimates Lower Limit Upper Limit    Gradient
--------------------------------------------------------------------------
         phi      0.9810     -0.8449     -0.6672     -0.0026
       theta     -0.6266      0.5200      1.0881      0.0005
      sigma2      6.5535      2.3061      2.8138      0.0000

Model and residual diagnostics:
==========================================================================

Ljung-Box (Q):                                                        4.12
Prob(Q):                                                            0.0425
Heteroskedasticity (H):                                               1.91
Prob(H):                                                           0.00885
Jarque-Bera (JB):                                                      616
Prob(JB):                                                        1.62e-134
Skew:                                                                 -1.3
Kurtosis:                                                             11.2
==========================================================================

Conclusion

State-space models are powerful and can be used to tackle a wide variety of modeling problems. With a little practice, state-space models can expand your modeling universe substantially.

This blog uses a simple $ARMA(1,1)$ model of inflation to give you a foundation for getting started with state-space models using the GAUSS sslib.

Further Reading

1. Filtering Data With the Kalman Filter
2. Beginner's Guide to Maximum Likelihood Estimation
3. Maximum Likelihood Estimation in GAUSS
4. Introduction to the Fundamentals of Time Series Data and Analysis
5. Getting Started With Time Series in GAUSS

Eric( Director of Applications and Training at Aptech Systems, Inc. )

Eric has been working to build, distribute, and strengthen the GAUSS universe since 2012. He is an economist skilled in data analysis and software development. He has earned a B.A. and MSc in economics and engineering and has over 18 years of combined industry and academic experience in data analysis and research.