4
$\begingroup$

I"m trying to see the big picture and where time series fits in to statistical inference.

I'm trying to understand when we would use a time-series model like ARIMA, GARCH, and others. From the studying I have done so far, it seems like time series models are best used to model the residuals created by other models. Not only that, it is only applicable to model residuals for very specific other models, like linear regression. In other words, broadly speaking, the models described in Tsay and Shumway's book seem to have very limited scope in real life application.

For example, if I were to build a boosting tree or neural network, I would imagine there is no need for any of the traditional time series models even if I was building the model on time series data? As neither a tree nor a neural network make any assumptions about the distribution of the data nor derive any analytical results, I can just include lagged values as another feature within the data. I could probably use the ACF plot or Ljung-Box test and look for serial correlation, then include the significant lags as additional features.

The models described in Tsay and Shumway, from my understanding, seem to only be applicable if your original model makes strict assumptions like normality and iid. Basically, it only applies to residuals of regression models. Otherwise I don't see how they could be of any use in real life - unless you just have some process that is 100% explained by lagged values of the same process.

Would you say my understanding is correct in that the models in Shumway and Tsay have limited scope in real life applications? This would make some sense as I rarely see job descriptions ask for one to understand time series models. There was a small burst maybe like 5 years ago asking for people to understand GARCH but now it's all about ML.

Thanks. I'm really having a hard time seeing how time series fits in with statistical inference and am hoping to finally clarify my understanding.

Right now every time I think of a "problem", I automatically think first to use a boosting tree (ease of use and it provides variable importance and has good performance), second to use neural networks (if I want to batch process, sequentially add data, or huge data sets), and third to use linear regression (if I want to know how much my feature affects the predictor, analytical results, and speed). I can't think of where TS comes in other than modeling my residuals of regression models.

$\endgroup$
12
  • 3
    $\begingroup$ Every model that is used for time series data is a time series model. "Time series" is not a type of solution, it is a type of data. Regardless, models which are too flexible tend to perform poorly when the flexibility is unnecessary or when insufficient data is available to support it. More strongly structured models like ARIMA or GARCH are in widespread every day use in Economics and Finance. It's not particularly surprising that an introductory textbook is limited to linear Gaussian models but the statistical literature on time series models is vastly more complete than that. $\endgroup$ – Chris Haug Jun 5 '20 at 21:37
  • 1
    $\begingroup$ Hi: machine learning is one type of broad technique for prediction. Time-series models such as ARIMA modelling, distributed lags, and kalman filtering approaches are another. I wouldn't classifiy all time-series models as regression models because the devil is in the details. There have been attempts to use say neural networks on time-series data but I'm not up on how successful they've been. In the end, whether you use ML or the more traditional time-series modelling approaches, you're eventually maximizing a likelihood so I don't see the two approaches as being all that far from each other. $\endgroup$ – mlofton Jun 5 '20 at 23:00
  • $\begingroup$ @mlofton Well I didn't mean that all time series models are regression models. I meant the residuals of your regular regression usually is normally distributed, identically distributed, and stationary, which fits the assumptions of some traditional time series models. I guess I can't think of a situation where I would use a traditional time series model as my first approach to modeling. Seems like I would only use it after I have built another model and see that the residuals of that model have some characteristics that can be modeled using a time series model. $\endgroup$ – confused Jun 6 '20 at 5:53
  • $\begingroup$ I haven't really examined the residuals of boosting trees or neural networks, but I would imagine that you can remove serial correlation by just including the relevant lagged terms as additional predictors and there wouldn't be an issue since the models don't require any strong assumptions. $\endgroup$ – confused Jun 6 '20 at 5:56
  • $\begingroup$ Note that it is often the case in time series modelling that you do not have any exogenous variables, you simply have observations from the univariate time series. You can still use any model you want, but the lagged terms will be your only input. Additionally, if you do have informative exogenous variables, you will need to follow the protocol you laid out of regressing and then modelling the residuals with a time series model. But, as @richard-hardy pointed out, at least for the case of linear regression, estimating arima and regression coefficients is done in a single step. $\endgroup$ – Ryan Volpi Jun 6 '20 at 15:24
2
$\begingroup$

Would you say my understanding is correct in that the models in Shumway and Tsay have limited scope in real life applications?

Every model has limited applicability; classical time series models such as ARIMA and GARCH are no exception. However, their use has been extensive for decades and continues to be so. It is not because they are correct -- none of the models are -- but because they are useful, mainly in allowing to simulate future values of time series processes and delivering accurate point and density forecasts.

There are numerous solid academic journals within economics and finance who focus on time series analysis, and you will find plenty of ARIMA and GARCH models there. A couple of titles: "Journal of Time Series Analysis" and "Journal of Financial Econometrics".

Practitioners in finance use ARIMA-GARCH models extensively for risk modeling in financial markets (stock, derivative, commodity, foreign exchange markets). The popular software packages rugarch and rmgarch for R are built for that by someone who is a finance practitioner himself. The applicability of classical time series models is not limited to finance, however. The popular software package forecast for R contains automated functions such as ets, auto.arima and other developed for predicting objects as diverse as expenditures on drugs by the Australian government, numbers of tourist visiting the country, electricity prices and loads, etc.

From the studying I have done so far, it seems like time series models are <...> only applicable to model residuals for very specific other models, like linear regression.

The ubiquitous modeling of prices in financial markets with pure ARIMA-GARCH models should answer the question regarding whether classical time series models are only applicable for residuals from other models (the answer is no). Besides, there is no technical reason for not using time series models on residuals of models other than linear regression.

The models described in Tsay and Shumway, from my understanding, seem to only be applicable if your original model makes strict assumptions like normality and iid.

This is not an accurate representation of reality. As Chris Haug points out in his comment, It's not particularly surprising that an introductory textbook is limited to linear Gaussian models but the statistical literature on time series models is vastly more complete than that. E.g. the broad class of GARCH models produces nonnormal, non-i.i.d. outcomes.

Otherwise I don't see how they could be of any use in real life - unless you just have some process that is 100% explained by lagged values of the same process.

As I hinted above, this is often the best approximation of reality that is very challenging to improve upon wihtout overfitting and reducing forecast quality.

Now to address your title question:

Are time series models limited in real life application and are primarily used to model the residuals of another model?

It depends on the area of application. For logarithmic returns on stock prices, you would use ARIMA-GARCH directly. For numbers of tourists or price of electricity, you would include seasonal and perhaps other factors and specify an ARIMA-GARCH structure for the residual.

$\endgroup$
6
  • 1
    $\begingroup$ Hi thanks for the answer. So far my understanding is that GARCH and related models are used to model the "noise" term within ARIMA models (or just non constant variance of your residuals). So far the models I have seen seem to be univariate as you are just predicting future values with lagged values. I haven't finished the books yet so things may change. What would be a model that incorporates additional features (not just lagged variables of itself)? The only way I know of incorporating other features is to build a model with other features first then use time series model on the residuals. $\endgroup$ – confused Jun 6 '20 at 11:33
  • 1
    $\begingroup$ For stock prices, the best model is often pure ARIMA-GARCH, as the data is mainly "noise" as you might call it. And yes, perhaps the easiest way to incorporate additional features is to do regression with ARIMA errors such as implemented in arima and auto.arima functions in R. However, the models are not sequential; they are specified and estimated in one step. It is for the purposes of pedagogical or estimation simplicity (which comes at the cost of efficiency) that you can consider them as first a model with additional features and second an ARIMA(-GARCH) model for the residuals. $\endgroup$ – Richard Hardy Jun 6 '20 at 12:23
  • 1
    $\begingroup$ @confused, There are also transfer function models that look like ARIMA but have exogenous regressors in them, and there it is a little harder to disentagle the regressors from the ARIMA part. For GARCH models specifically, the equation for $\sigma^2_t$ can contain exogenous regressors representing seasonality and any other factors. It is still a GARCH model, but it contains information from several time series, and it can be fit directly on raw data. $\endgroup$ – Richard Hardy Jun 6 '20 at 12:25
  • $\begingroup$ I don't understand why for stock prices the best model is ARIMA-GARCH. Is the goal purely to model the movement of stock prices as opposed to prediction? I would imagine that if prediction was the goal, one would want to include exogenous variables - such as creating some sort of factor model. I'll also have to look into GARCH models again and it allowing for exogenous variables. $\endgroup$ – confused Jun 7 '20 at 22:05
  • 1
    $\begingroup$ @confused, stock price prediction is notoriously difficult. Your idea of including exogenous variables is intuitive (though you have to remember the bias-variance trade-off), but practice shows that beating the forecast from a simple ARIMA-GARCH model is very difficult (basically, because of the trade-off). $\endgroup$ – Richard Hardy Jun 8 '20 at 5:24
1
$\begingroup$

To complement Richard Hardy's answer, statistical time series models like ARIMA and the various members of the exponential smoothing family are used extensively in Supply Chain management and Retail to forecast demand, capacity, and inventory needs. And no, they are not merely modeling the residuals of some other supervised learning model, but modelling 1000s, or sometimes 1000000s of time series directly (consider for example a retailer with 500 stores and 50000 individual products: you end up 2.5M time series).

Although the various R time series packages are useful and pedagogical, they typically don't scale well for such applications. Organizations resort to "industrial grade" large scale ERP packages, or more recently, building custom forecasting solutions on the cloud.

It's ironic that you mention neural networks and tree-based methods as more useful or versatile, because in the time series forecasting space, those are considered relative newcomers who have yet to prove their usefulness. They're not entirely useless, there's just a debate around how useful they are compared to the statistical workhorses of forecasting, which are the various members of the exponential smoothing family.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.