1
$\begingroup$

I'm modeling time series using dynamic linear/state-space models, and was surprised by how inaccurate the estimates of model parameters can be, even for fairly long series.

In particular, I'm generating series of length n using the second-order local linear trend model:

$\begin{align} y_t &= \mu_t + \alpha_t & \alpha_t \sim N(0, \sigma_\alpha^2)\\ \mu_{t+1} &= \mu_t + \nu_t + \beta_t & \beta_t \sim N(0, \sigma_\beta^2)\\ \nu_{t+1} &= \nu_t + \gamma_t & \gamma_t \sim N(0, \sigma_\gamma^2) \end{align}$

for some particular values of the the three variances $\sigma_\alpha^2, \sigma_\beta^2$, and $\sigma_\gamma^2$. Here the observed series is $y_t$, while $\mu_t$ and $\nu_t$ are unobserved state variables. I then estimate the variances using the $\texttt{dlm}$ package in R.

Here is some code:

library(dlm)

n <- 100
v <- c(0.1, 0.2, 0.3)
s <- sqrt(v)
mu <- rnorm(1)
nu <- rnorm(1)
y <- vector(mode = "numeric", length = n)
for(i in 1:n) {
    y[i] <- mu + rnorm(1, sd = s[1])
    mu <- mu + nu + rnorm(1, sd = s[2])
    nu <- nu + rnorm(1, sd = s[3])
}

my_build <- function(p) {
    dlmModPoly(order = 2, dV = p[1], dW = c(p[2], p[3]))
}
mle <- dlmMLE(y, parm = c(1, 1, 1), build = my_build, lower = c(1e-4, 1e-4, 1e-4))
print(mle$par)

(Sorry for the probably bad R code, I'm just a beginner R programmer.) It generates a $n=100$ series according to the above equations with variances equal to 0.1, 0.2, and 0.3, and then estimates the variances using the dlmMLE function.

What surprised me is how off these estimates can be, even for a series of length 100. The average absolute error is about 0.11 (so around 50%). For a series with $n=1000$, the error is still around 0.04 (so about 20%).

Is this normal? Is parameter estimation for state-space models inherently difficult and noisy and error-prone (and therefore requires very long series for accurate estimates), or am I doing something wrong?

$\endgroup$
2
$\begingroup$

"Is this normal?" I don't use this software, and I'm not sure what you mean by "normal," but nothing jumps out at me. A few thoughts:

  1. Asymptotic distributions of MLE estimates have a variance term that is the reciprocal of the Fisher Information. This means, roughly, that you get wider confidence intervals when the likelihood surface is "flatter" or "less pinched" on average. This has connections with how identifiable a model is. In other words, not all SSMs are created equally. I always think about how much I could pare down a model by removing a parameter or two, and if that would make a difference.

I'm not sure how you're calculating these intervals, but it's all just food for thought.

  1. Asymptotics only apply when your data set is large enough. Your data might not be large enough to justify these kind of confidence intervals. In this case, I'd recommend either collecting more data, or transforming the parameter space so that MLE distributions become more normal. You could start off by estimating $\log \sigma^2$ or $\log \sigma$ instead of $\sigma^2$. If you exponentiate that parameter estimate, that would give you the estimate of $\sigma^2$, by the invariance principle. This would also free you from having to specify lower bounds for your algorithm.

  2. You're talking about percentage errors, which seems like you're interested in estimating $\sigma^2/ \mu$. You can do this with standard MLE theory, as well, using the invariance property again, and getting confidence intervals using the delta method, etc.

  3. Try plotting profile log-likelihoods. Hold one of the three parameters fixed, simulate only one data set, and see if this surface looks flat at the top.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.