tl;dr: linear model is better than ANN and decision tree in timeseries regression task, why is that?
I have a time series dataset of 151 observations each with 43 macroeconomic variables. Some of the variables are percentage change from year over year (e.g. GDP change), and some of them are stationary (I hope I use that word correctly e.g. employment rate). The frequency of data is quarterly. Variable I want to predict is CPI inflation and it is lagged appropriately t-1. I divide the data set to training and testing in many different proportions.
My problem is that multiple linear regression performs better ( as of MSE and R squared) than machine learning techniques like: artificial neural networks, decision trees with and without extreme gradient boosting. They of course both performed well but linear regression is always better MSE 0.2 vs 0.8 and r squared 87% and 82%. In the case of ANN I tried training them in almost every possible way, I guess. the most confusing thing is that ANN according to Universal approximation theorem should performed at least as good as linear regression. Maybe the linear regression I make is misleading as the following warning is being shown "prediction from a rank-deficient fit may be misleading". Also when I compare logistic regression and these ML techniques in classification task, they performed better
Anyone have experienced such an outcome? Is it normal? and if not how can I repair that?
I use R language and package neuralnets and xgBoost.