3
$\begingroup$

I have fitted a SARIMA model to my time series. The diagnostics of the residuals are all good (ACF, PACF, ...), i.e. it seems they behave like white noise.

But when I plot the normal qq-plot, they seem to be asymmetrically distributed. I tried doing a Student's $t$ qq-plot and the errors seems to be corrected. My only problem is that now, I do not know how to interpret this result.

What does it mean for the residuals to follow a Student's $t$ distribution?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ what is the context? what is the application? this could mean many different things depending on your problem $\endgroup$
    – Aksakal
    Commented Jul 8, 2016 at 1:11

2 Answers 2

Reset to default
2
$\begingroup$

If the residuals appear to be asymmetric judging by the normal Q-Q plot, it surprises me that they no longer appear asymmetric judging by the Student's $t$ Q-Q plot, because both the Normal and the Student's $t$ distributions are symmetric.

Aside of that, you can say you have fit your SARIMA model using quasi maximum likelihood (QML) rather than maximum likelihood (ML), as the realized error distribution does not match the assumed error distribution. Since both Normal and Student's $t$ distributions are in the same exponential family, the QML coefficient estimators are consistent. Just the standard errors should be adjusted to reflect the mismatch in distributions.

Improve this answer
$\endgroup$
13
  • $\begingroup$ Please explain in greater detail. For example, if $1<v\leq 2$ then the second moment (standard deviation squared) of a Student's-$t$ is not defined. I would in that case use jackknife of 1/2 of the number of samples 1000 times to establish confidence intervals, which would not depend on a distribution assumption at all, what would you do? $\endgroup$
    – Carl
    Commented Jul 9, 2016 at 17:15
  • $\begingroup$ I don't see how your comment is related to my answer. It might be an interesting question in itself, though, I do not argue about that. Regarding degrees of freedom: heavy tails may be problematic, and I am not sure how well QML works in case of Student-$t$ with $t<2$. One should check the assumptions for QML to work to be sure. $\endgroup$
    – Richard Hardy
    Commented Jul 9, 2016 at 20:22
  • $\begingroup$ A few more notes. First, jackknife, bootstrap and related techniques cannot be straightforwardly applied to time series where the time dimension plays a role; and especially to (S)ARIMA models where model parameters are highly nonlinear functions of errors. Second, jackknife (unlike boostrap subsampling) is done once (not 1000 times) for a fixed sample. Third, variance (squared standard deviation) is second central moment. $\endgroup$
    – Richard Hardy
    Commented Jul 9, 2016 at 20:49
  • $\begingroup$ OK, central moment. Easy to show linear case, SARIMA residuals, if they are Student's-$t$ do not care what they came from, they are what they are. What I have done is to point the way to back calculating them, by using Monte Carlo simulation. Jackknife after 1000 resamplings of 206 objects from a collection of 412 is likely enough, doing it once would be impossible for the $1.163427960939212\times 10^{511}$ resamplings, what what I was getting at. $\endgroup$
    – Carl
    Commented Jul 10, 2016 at 7:05
  • $\begingroup$ The default model selection criterion in Mathematica 10.3 for their TimeSeriesModelFit that takes SARMA (seasonal autoregressive moving-average model family) is AIC. I do not know what was used for the question above. Maximum likelihood does not appear to DO anything differently than minimizing SSR (sum squared residuals), which is OLS. Maybe there is some subtle difference that you could explain to me. In practice, to use AIC for model selection may require more random searching and lots of it for non-normal conditions. Consider using other criteria for model selection and then compare. $\endgroup$
    – Carl
    Commented Aug 2, 2016 at 4:39
2
$\begingroup$

What does it mean for the residuals to follow a Student's t distribution?

The meaning depends on the degrees of freedom, $v$, of the Student's-t Distribution. For background information the Wikipedia entries on the Cauchy, Student's-$t$, and normal distributions are useful reading. I have not seen anything theoretically quantify the regression errors for the Student's-$t$ distribution from ordinary least squares (OLS) regression. For maximum likelihood iff degrees of freedom > 2 one can use MatLab, also see ref. Cross-Validated. It is easy enough to show what the effects are using simulations.

For 1 degree of freedom, the Student's-t becomes a Cauchy distribution and OLS regression is not useful, because the mean of a Cauchy distribution is undefined (therefore unstable when calculated). See the first image, Monte Carlo simulation of 10000 Cauchy residuals for sequential equidistant x-axis points, below with regression equation $y=3.50054 x-19394.7$ where the truth is $y=x$ Monte Carlo simulation of 10000 Cauchy residuals for sequential equidistant x-axis points

When the degrees of freedom are greater than 1 up to and including 2, the mean is defined but the standard deviation (second moment) is not defined, and then the OLS regression is more accurate but not exact, see next image with $v=2$, and regression equation $1.04074 x-337.02$. <span class=$v=2$" />

As $v$ increases the regression accuracy does as well. At $v>3$, the third moment, skewness is defined. For $v>4$ kurtosis is defined, and so on for higher moments and degrees of freedom.

To get some idea of what this means in practice, I made sequential models for increasing $v$ on the $x$-axis using two sets of random numbers, which coincidentally approach a slope of one and intercept of zero from opposite directions. First the intercepts:Intercepts for increasing df

Next the slopes:slopes for increasing df

As $v\to \infty$ the Student's-$t$ distribution becomes a normal distribution. Thank-you for a very interesting question that I was wondering about myself.

Improve this answer
$\endgroup$
13
  • 1
    $\begingroup$ While a nice answer, I think it misses the point. The question is not what realizations of a Student-$t$ distributed random variable look like but rather what does that mean in the context of (residuals of) SARIMA models. But let the OP decide how relevant that is. $\endgroup$
    – Richard Hardy
    Commented Jul 9, 2016 at 20:24
  • $\begingroup$ The model residuals are Student's-$t$, exactly. The question was what is the effect, and, this is the easiest way of showing it. How one obtains residuals from more complicated models, like SARIMA models is rather besides the point, and the analysis of raw residuals does not much care what they came from. I get your point, if you have a better answer, I would be delighted to see it. $\endgroup$
    – Carl
    Commented Jul 10, 2016 at 6:18
  • $\begingroup$ Sorry for not being specific enough. The point is not how to obtain residuals from a model. It is what effect the mismatch between assumed and realized distributions (for errors and residuals, respectively) have on maximum likelihood estimates and, consequently, inference and forecasts from the model. I posted an answer for that on May 22. $\endgroup$
    – Richard Hardy
    Commented Jul 10, 2016 at 7:25
  • $\begingroup$ I want to understand your answer better, it would help if you explained your answer in greater detail. My answer, use Monte Carlo simulation on whatever the data and model are has the advantage of being simple, and intuitive. $\endgroup$
    – Carl
    Commented Jul 10, 2016 at 8:24
  • 2
    $\begingroup$ My answer has two parts: one regarding asymmetry, another regarding MLE and QMLE. I suppose you are interested in the second part. It is nothing special, just an application of the idea/principle of QMLE introduced in White, Halbert. "Maximum likelihood estimation of misspecified models." Econometrica: Journal of the Econometric Society (1982): 1-25. (There is also a reference to a nice treatment in Wooldridge's textbook in this thread.) $\endgroup$
    – Richard Hardy
    Commented Jul 10, 2016 at 10:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.