tl;dr: linear model is better than ANN and decision tree in timeseries regression task, why is that?
I have a time series dataset of 151 observations each with 43 macroeconomic variables. Some of the variables are percentage change from year over year (e.g. gdp change), and some of them are stationary (i hope i use that word correctly e.g. employement rate). The frequency of data is quarterly. Variable i want to predict is CPI inflation and it is lagged appropriately t-1. i divide the data set to training and testing in many diffrent proportions.
My problem is that mutlivariate linear regression performes better ( as of MSE and R squared) than machine learning techinques like: artificial neural networks, decision trees with and without extreme gradient boosting. They of course both performe well but linear regression is always better MSE 0.2 vs 0.8 and r squared 87% and 82%. In the case of ANN i tried training them in almost every possible way, i guess. the most confusing thing is that ANN according to Universal approximation theorem should performe at least as good as linear regression. Maybe the linear regression i make is misleading as the following warning is being shown"prediction from a rank-deficient fit may be misleading". Also when i compare logistic regression and these ML techniques in classification task, they performe better
Anyone have experienced such an outcome? Is it normal? and if not how can i repair that?
I use R language and package neuralnets and xgBoost. If anything is unclear i can provide aditional informations. thanks in advance!
