I see many online blogs that use Linear Regression to predict future stock price. I too have done this and my x variable is time elapsed.
But I have been advised this problem is better suited for Time Series problem instead of Linear Regression. I want to understand why and what is the difference between say ARIMA and Linear Regression in the context of predicting future stock prices based on historical data. e.g. date and closing price.
I want to understand why and what is the difference between say ARIMA and Linear Regression in the context of predicting future stock prices based on historical data. e.g. date and closing price.
Usually in financial literature "stock predictability" should be intended as predictability of stock returns, not stock price. Indeed, predict price variations is what is challenging and potentially fruitful.
For (log)price model like ARIMA(p,1,q) are usual, but them imply an ARMA(p,q) for (log)return. So after this passage we can work with ARMA (subclass of ARIMA).
Thsi discussion is related: Forecasting Prices vs Returns by Deep Learning
Now, for returns predictability we can consider linear regression and ARMA models as competitors. It seems me that your question is about that.
In some presentations ARMA can appear as separate concept respect to linear regression. Even the background matters, several machine learners never face ARMA models. However seems me very useful to grasp the connections at first place.
Indeed them are strongly related. Suffice to keep in mind that under usual condition (stationarity, invertibility) an ARMA process can be represented as a pure AR one, pure autoregressive. However for parsimony/efficiency reasons ARMA specifications can be better; if MA part is included we move away from usual "OLS regression".
Use past returns as predictors is frequently the first step even for "regressionist". Then we can say that even them performs ARMA models (a subclass of them), sometimes unknowingly.
Moreover both (linear regression and ARMA) can be considered as specifications for conditional expectation function (CEF) in linear approximation. (Related: Regression and the CEF)
The main difference between the two is that ARMA models consider only past values of the serie under analysis, while linear regression is more general and permit to consider other variables as predictors.
Finally, in estimation term for ARMA the ML procedure are usual. For linear regression we can use OLS but even procedure as LASSO or RIDGE can be involved.
