# How to handle incomplete data in Kalman Filter?

What are some typical approaches to handling incomplete data in the Kalman Filter? I'm talking about the situation where some elements of the observed vector $y_t$ are missing, distinct from the case where an entire observed vector $y_t$ is missed. Another way of thinking about this would be that the dimension $p$ of the observed vector is different for each time point.

To explain my context a bit further, the observations are estimated parameters from a logistic regression performed at each time point. While each logistic regression includes the same covariates, sometimes the estimates are undefined due to collinearities in the data for that time point.

• I suggest you this simply python implementation stackoverflow.com/a/63630858/10375049 Aug 28 '20 at 9:37
• @MarcoCerliani, the question you linked to deals with missing time steps, rather than incomplete data for a given time step, right? Oct 25 '21 at 0:40

What is needed is simply to have a variable observation matrix, i.e. in the observation equation: $$\boldsymbol{Y_t} = \boldsymbol{A_t}\boldsymbol{\theta_t} + \boldsymbol{R_t}\boldsymbol{e_t}$$ matrix $\boldsymbol{A_t}$ (and $\boldsymbol{R_t}$) should omit at time $t$ the rows corresponding to NA entries in $\boldsymbol{Y_t}$. Most packages in R, for instance, will take care of that: you can have the observed multivariate time series with NA values without problems.
• Would like to bump up this best answer for attention. I read the book Time Series Analysis and Its Applications: With R Examples by Robert Shumway, and in chapter 6, the author described the way of handling missing observations in $Y_t$ is by filling with zeros. How does your method of row/column elimination compare with this method? Jan 12 '17 at 17:27
• I think both methods are equivalent. Note, however, that what Shumway & Stoffer propose (and claim it is computationally simpler; I guess it depends on which software you are using) requires filling with zeros not only $Y_t$ but other arrays as well (check their page 347, just above (6.79) of the third edition). Jan 15 '17 at 13:31
• For those who don't have access to Shumway: You have to 0 out the missing observation rows in $Y$ and $A$ (your residual vector and observation model matrix), but replace the missing parts of $R$ (observation covariance matrix) with a component identity matrix (1's along the missing diagonals, and 0's in the missing covariances). Oct 26 '21 at 20:37