3
$\begingroup$

In the "Dynamic Linear Models with R" book, the Regression models section reads: "The static regression linear model corresponds to the case where $W_t = 0$ for any $t$, so that $\theta_t = \theta$ is constant over time."

I am not understanding what this means, because when I fit a dlmModReg with dW = 0 (default values) and then plot the predicted values of the state vectors, my regression coefficients vary over time, and eventually stabilize to a value similar to that of standard least-squares regression.

Plot of regression slope coefficient with dW = 0:

dW = 0

However, if I use dlmMLE to find the MLE of dW before fitting the model, plotting the predicted values of the state vectors results in non-sensible values with multiple large discontinuities.

Plot of regression slope coefficient with dW MLE and no intercept:

dW != 0 no intercept

Plot of regression slope with dW MLE with intercept: dw != 0 with intercept

I've yet to find any literature on how the intercept coefficient gets set, but I'm observing a near perfect linear relationship between the intercept and slope coefficient. The inclusion of an intercept term in the parameter MLE also causes my dV to drop from 16 to 0.003. Can anyone point me to any references which discuss how the intercept term is set on each update?

Plot of intercept and slope with dW MLE: intercept and slope

My questions are:

  1. Why are the regression coefficients dynamic/time-varying even with dW = 0 in the first example? If the regression coefficient changes at every time step regardless of whether dW is 0 or not due to the measurement update stage $\beta_t = \hat{\beta_t} + K_t(y_t - \alpha_t - \hat{\beta_t}x_t)$ ($K$ is the Kalman gain), what's the difference between simple linear regression and dynamic linear regression except for some random variation added in the time update equation $\hat{P_t} = P_{t-1} + dW$ ($P$ is estimate error covariance)?
  2. Why does the plot of regression coefficients have so many discontinuities when my dW != 0?
  3. What is the relationship between the intercept and the slope coefficients? Plotting them reveals a near-perfect linear relationship, but I can't find any literature explaining this. I haven't found the intercept term included in any formulation of the Kalman update equations.

Would anyone be willing to take a look at my data/code?

$\endgroup$
  • $\begingroup$ The linear relationship only holds when dW = 0, so I think it's just how the intercept/slope evolve over time when performing static linreg at each data point. For now I'm interested in getting this to work with dW != 0 so I'm going to ignore question 3 for now. $\endgroup$ – tmakino May 30 '13 at 14:18
1
$\begingroup$

I cannot answer 2 and 3; the fact that there is near-perfect relationship among slope ant intercept coefficients suggest that the variance of both may be quite large and also account for the discontinuities you observe in 2. But one would need to look at your data and code.

Your question 1, on the other hand, is easy. You have to realize that, even if you assume the parameters are fixed, the Kalman filter gives a sequence of estimates with observations up to a certain time point t. If you were to fit ordinary regressions with samples from 1 to t for different t, you would also obtain different estimates with each.

What is true is that, except for numerical problems that may arise (and for the possible effect of the prior), the final estimate of the state would coincide with the estimate obtained from the ordinary regression with the same sample.

$\endgroup$
  • $\begingroup$ If running Kalman filter with $dW = 0$ is equivalent (yet more computationally efficient) to running OLS at each time step on the entire data set, is there an analogy which will help me understand what "dynamic regression" is, namely where $dW != 0$? I've written the Kalman filter equations out by hand and the only difference is in the time update equations, when the a priori estimate error is updated as the sum of the a posteriori estimate error from the previous time step, and dW. $\endgroup$ – tmakino May 29 '13 at 21:59
  • $\begingroup$ @tmakino: If you use the Kalman filter with dW != 0 you are actually allowing the parameters to change. The closest analogy I can think of is running a regression on overlapping windows from t to (t+k), for t varying, so different betas are fitted in each interval. $\endgroup$ – F. Tusell May 30 '13 at 15:02
  • $\begingroup$ @FTusell: I want to perform dynamic regression, and I've been going line by line in the "Dynamic Linear Models with R" book and also your paper "Kalman Filtering in R" and have had no luck yet. It seems like as long as dW != 0 my regression coefficients don't converge, and so far I've tried both the MLE estimates via dlmMLE, and putting in values manually to test. It seems like as long as dV != 0 it doesn't matter whether its set to it's MLE or almost any other manually entered value, it still converges as long as dW != 0. I'm not sure what to do at this point. $\endgroup$ – tmakino May 30 '13 at 21:00
  • $\begingroup$ @FTusell: I'm also having difficulty finding any references which show the necessary calculations when both the intercept and slope are allowed to be time-varying. There's a code example in the book I mentioned above on how to set up the dlm model, but no equations. $\endgroup$ – tmakino May 30 '13 at 21:00
  • $\begingroup$ You say "It seems like as long as dW != 0 my regression coefficients don't converge" and then "it still converges as long as dW != 0". I gather the last one is meant "it still does not converge as long as dW != 0". But what do you expect them to converge to, if you allow time variation (and thus assume they are not constant)? $\endgroup$ – F. Tusell May 31 '13 at 10:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.