Are time series models limited in real life application and are primarily used to model the residuals of another model? I"m trying to see the big picture and where time series fits in to statistical inference.  
I'm trying to understand when we would use a time-series model like ARIMA, GARCH, and others.  From the studying I have done so far, it seems like time series models are best used to model the residuals created by other models.  Not only that, it is only applicable to model residuals for very specific other models, like linear regression.  In other words, broadly speaking, the models described in Tsay and Shumway's book seem to have very limited scope in real life application.  
For example, if I were to build a boosting tree or neural network, I would imagine there is no need for any of the traditional time series models even if I was building the model on time series data?  As neither a tree nor a neural network make any assumptions about the distribution of the data nor derive any analytical results, I can just include lagged values as another feature within the data.  I could probably use the ACF plot or Ljung-Box test and look for serial correlation, then include the significant lags as additional features.  
The models described in Tsay and Shumway, from my understanding, seem to only be applicable if your original model makes strict assumptions like normality and iid.  Basically, it only applies to residuals of regression models.  Otherwise I don't see how they could be of any use in real life - unless you just have some process that is 100% explained by lagged values of the same process.
Would you say my understanding is correct in that the models in Shumway and Tsay have limited scope in real life applications?  This would make some sense as I rarely see job descriptions ask for one to understand time series models.  There was a small burst maybe like 5 years ago asking for people to understand GARCH but now it's all about ML. 
Thanks.  I'm really having a hard time seeing how time series fits in with statistical inference and am hoping to finally clarify my understanding.  
Right now every time I think of a "problem", I automatically think first to use a boosting tree (ease of use and it provides variable importance and has good performance), second to use neural networks (if I want to batch process, sequentially add data, or huge data sets), and third to use linear regression (if I want to know how much my feature affects the predictor, analytical results, and speed).  I can't think of where TS comes in other than modeling my residuals of regression models.
 A: 
Would you say my understanding is correct in that the models in Shumway and Tsay have limited scope in real life applications? 

Every model has limited applicability; classical time series models such as ARIMA and GARCH are no exception. However, their use has been extensive for decades and continues to be so. It is not because they are correct -- none of the models are -- but because they are useful, mainly in allowing to simulate future values of time series processes and delivering accurate point and density forecasts.
There are numerous solid academic journals within economics and finance who focus on time series analysis, and you will find plenty of ARIMA and GARCH models there. A couple of titles: "Journal of Time Series Analysis" and "Journal of Financial Econometrics".
Practitioners in finance use ARIMA-GARCH models extensively for risk modeling in financial markets (stock, derivative, commodity, foreign exchange markets). The popular software packages rugarch and rmgarch for R are built for that by someone who is a finance practitioner himself. The applicability of classical time series models is not limited to finance, however. The popular software package forecast for R contains automated functions such as ets, auto.arima and other developed for predicting objects as diverse as expenditures on drugs by the Australian government, numbers of tourist visiting the country, electricity prices and loads, etc.

From the studying I have done so far, it seems like time series models are <...> only applicable to model residuals for very specific other models, like linear regression.

The ubiquitous modeling of prices in financial markets with pure ARIMA-GARCH models should answer the question regarding whether classical time series models are only applicable for residuals from other models (the answer is no). Besides, there is no technical reason for not using time series models on residuals of models other than linear regression.

The models described in Tsay and Shumway, from my understanding, seem to only be applicable if your original model makes strict assumptions like normality and iid. 

This is not an accurate representation of reality. As Chris Haug points out in his comment, It's not particularly surprising that an introductory textbook is limited to linear Gaussian models but the statistical literature on time series models is vastly more complete than that. E.g. the broad class of GARCH models produces nonnormal, non-i.i.d. outcomes.

Otherwise I don't see how they could be of any use in real life - unless you just have some process that is 100% explained by lagged values of the same process.

As I hinted above, this is often the best approximation of reality that is very challenging to improve upon wihtout overfitting and reducing forecast quality.
Now to address your title question:

Are time series models limited in real life application and are primarily used to model the residuals of another model?

It depends on the area of application. For logarithmic returns on stock prices, you would use ARIMA-GARCH directly. For numbers of tourists or price of electricity, you would include seasonal and perhaps other factors and specify an ARIMA-GARCH structure for the residual.
A: To complement Richard Hardy's answer, statistical time series models like ARIMA and the various members of the exponential smoothing family are used extensively in Supply Chain management and Retail to forecast demand, capacity, and inventory needs. And no, they are not merely modeling the residuals of some other supervised learning model, but modelling 1000s, or sometimes 1000000s of time series directly (consider for example a retailer with 500 stores and 50000 individual products: you end up 2.5M time series).
Although the various R time series packages are useful and pedagogical, they typically don't scale well for such applications. Organizations resort to "industrial grade" large scale ERP packages, or more recently, building custom forecasting solutions on the cloud.
It's ironic that you mention neural networks and tree-based methods as more useful or versatile, because in the time series forecasting space, those are considered relative newcomers who have yet to prove their usefulness. They're not entirely useless, there's just a debate around how useful they are compared to the statistical workhorses of forecasting, which are the various members of the exponential smoothing family.
