I have fitted a SARIMA model to my time series. The diagnostics of the residuals are all good (ACF, PACF, ...), i.e. it seems they behave like white noise.

But when I plot the normal qq-plot, they seem to be asymmetrically distributed. I tried doing a Student's $t$ qq-plot and the errors seems to be corrected. My only problem is that now, I do not know how to interpret this result.

What does it mean for the residuals to follow a Student's $t$ distribution?

  • 1
    $\begingroup$ what is the context? what is the application? this could mean many different things depending on your problem $\endgroup$ – Aksakal Jul 8 '16 at 1:11

What does it mean for the residuals to follow a Student's t distribution?

The meaning depends on the degrees of freedom, $v$, of the Student's-t Distribution. For background information the Wikipedia entries on the Cauchy, Student's-$t$, and normal distributions are useful reading. I have not seen anything theoretically quantify the regression errors for the Student's-$t$ distribution from ordinary least squares (OLS) regression. For maximum likelihood iff degrees of freedom > 2 one can use MatLab, also see ref. Cross-Validated. It is easy enough to show what the effects are using simulations.

For 1 degree of freedom, the Student's-t becomes a Cauchy distribution and OLS regression is not useful, because the mean of a Cauchy distribution is undefined (therefore unstable when calculated). See the first image, Monte Carlo simulation of 10000 Cauchy residuals for sequential equidistant x-axis points, below with regression equation $y=3.50054 x-19394.7$ where the truth is $y=x$ Monte Carlo simulation of 10000 Cauchy residuals for sequential equidistant x-axis points

When the degrees of freedom are greater than 1 up to and including 2, the mean is defined but the standard deviation (second moment) is not defined, and then the OLS regression is more accurate but not exact, see next image with $v=2$, and regression equation $1.04074 x-337.02$. $v=2$

As $v$ increases the regression accuracy does as well. At $v>3$, the third moment, skewness is defined. For $v>4$ kurtosis is defined, and so on for higher moments and degrees of freedom.

To get some idea of what this means in practice, I made sequential models for increasing $v$ on the $x$-axis using two sets of random numbers, which coincidentally approach a slope of one and intercept of zero from opposite directions. First the intercepts:Intercepts for increasing df

Next the slopes:slopes for increasing df

As $v\to \infty$ the Student's-$t$ distribution becomes a normal distribution. Thank-you for a very interesting question that I was wondering about myself.

  • 1
    $\begingroup$ While a nice answer, I think it misses the point. The question is not what realizations of a Student-$t$ distributed random variable look like but rather what does that mean in the context of (residuals of) SARIMA models. But let the OP decide how relevant that is. $\endgroup$ – Richard Hardy Jul 9 '16 at 20:24
  • $\begingroup$ The model residuals are Student's-$t$, exactly. The question was what is the effect, and, this is the easiest way of showing it. How one obtains residuals from more complicated models, like SARIMA models is rather besides the point, and the analysis of raw residuals does not much care what they came from. I get your point, if you have a better answer, I would be delighted to see it. $\endgroup$ – Carl Jul 10 '16 at 6:18
  • $\begingroup$ Sorry for not being specific enough. The point is not how to obtain residuals from a model. It is what effect the mismatch between assumed and realized distributions (for errors and residuals, respectively) have on maximum likelihood estimates and, consequently, inference and forecasts from the model. I posted an answer for that on May 22. $\endgroup$ – Richard Hardy Jul 10 '16 at 7:25
  • $\begingroup$ I want to understand your answer better, it would help if you explained your answer in greater detail. My answer, use Monte Carlo simulation on whatever the data and model are has the advantage of being simple, and intuitive. $\endgroup$ – Carl Jul 10 '16 at 8:24
  • 2
    $\begingroup$ My answer has two parts: one regarding asymmetry, another regarding MLE and QMLE. I suppose you are interested in the second part. It is nothing special, just an application of the idea/principle of QMLE introduced in White, Halbert. "Maximum likelihood estimation of misspecified models." Econometrica: Journal of the Econometric Society (1982): 1-25. (There is also a reference to a nice treatment in Wooldridge's textbook in this thread.) $\endgroup$ – Richard Hardy Jul 10 '16 at 10:09

If the residuals appear to be asymmetric judging by the normal Q-Q plot, it surprises me that they no longer appear asymmetric judging by the Student's $t$ Q-Q plot, because both the Normal and the Student's $t$ distributions are symmetric.

Aside of that, you can say you have fit your SARIMA model using quasi maximum likelihood (QML) rather than maximum likelihood (ML), as the realized error distribution does not match the assumed error distribution. Since both Normal and Student's $t$ distributions are in the same exponential family, the QML coefficient estimators are consistent. Just the standard errors should be adjusted to reflect the mismatch in distributions.

  • $\begingroup$ Please explain in greater detail. For example, if $1<v\leq 2$ then the second moment (standard deviation squared) of a Student's-$t$ is not defined. I would in that case use jackknife of 1/2 of the number of samples 1000 times to establish confidence intervals, which would not depend on a distribution assumption at all, what would you do? $\endgroup$ – Carl Jul 9 '16 at 17:15
  • $\begingroup$ I don't see how your comment is related to my answer. It might be an interesting question in itself, though, I do not argue about that. Regarding degrees of freedom: heavy tails may be problematic, and I am not sure how well QML works in case of Student-$t$ with $t<2$. One should check the assumptions for QML to work to be sure. $\endgroup$ – Richard Hardy Jul 9 '16 at 20:22
  • $\begingroup$ A few more notes. First, jackknife, bootstrap and related techniques cannot be straightforwardly applied to time series where the time dimension plays a role; and especially to (S)ARIMA models where model parameters are highly nonlinear functions of errors. Second, jackknife (unlike boostrap subsampling) is done once (not 1000 times) for a fixed sample. Third, variance (squared standard deviation) is second central moment. $\endgroup$ – Richard Hardy Jul 9 '16 at 20:49
  • $\begingroup$ OK, central moment. Easy to show linear case, SARIMA residuals, if they are Student's-$t$ do not care what they came from, they are what they are. What I have done is to point the way to back calculating them, by using Monte Carlo simulation. Jackknife after 1000 resamplings of 206 objects from a collection of 412 is likely enough, doing it once would be impossible for the $1.163427960939212\times 10^{511}$ resamplings, what what I was getting at. $\endgroup$ – Carl Jul 10 '16 at 7:05
  • $\begingroup$ The default model selection criterion in Mathematica 10.3 for their TimeSeriesModelFit that takes SARMA (seasonal autoregressive moving-average model family) is AIC. I do not know what was used for the question above. Maximum likelihood does not appear to DO anything differently than minimizing SSR (sum squared residuals), which is OLS. Maybe there is some subtle difference that you could explain to me. In practice, to use AIC for model selection may require more random searching and lots of it for non-normal conditions. Consider using other criteria for model selection and then compare. $\endgroup$ – Carl Aug 2 '16 at 4:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy