# Why are machine learning algorithms performing worse than standard multiple linear regression?

tl;dr: linear model is better than ANN and decision tree in timeseries regression task, why is that?

I have a time series dataset of 151 observations each with 43 macroeconomic variables. Some of the variables are percentage change from year over year (e.g. GDP change), and some of them are stationary (I hope I use that word correctly e.g. employment rate). The frequency of data is quarterly. Variable I want to predict is CPI inflation and it is lagged appropriately t-1. I divide the data set to training and testing in many different proportions.

My problem is that multiple linear regression performs better ( as of MSE and R squared) than machine learning techniques like: artificial neural networks, decision trees with and without extreme gradient boosting. They of course both performed well but linear regression is always better MSE 0.2 vs 0.8 and r squared 87% and 82%. In the case of ANN I tried training them in almost every possible way, I guess. the most confusing thing is that ANN according to Universal approximation theorem should performed at least as good as linear regression. Maybe the linear regression I make is misleading as the following warning is being shown "prediction from a rank-deficient fit may be misleading". Also when I compare logistic regression and these ML techniques in classification task, they performed better

Anyone have experienced such an outcome? Is it normal? and if not how can I repair that?

I use R language and package neuralnets and xgBoost.

• It is normal for linear regression to outperform other methods when the problem you have is well suited for that type of model. Universal approximation theorem doesn't mean that ANN will outperform any method on all problems. It means that ANN is flexible enough (given enough parameters) to approximate functions of arbitrary shape. If the dependencies in your data are truly linear then linear regression should outperform more complex models. Especially so on lower sample sizes. But I wouldn't ignore the warning you got, it basically says that you have more features than samples. Commented Sep 5, 2018 at 17:24
• Thanks for answer! it is rather frustrating but anyway i had to accept it haha. Althought do you think that even if my ANN will have single node and one hidden layer it could performe worse than linear regression? Commented Sep 5, 2018 at 17:33
• Your dataset is tiny. Typically advanced ML techniques start to outperform simpler techniques when you have a lot more data. Commented Sep 5, 2018 at 18:26
• @KarolisKoncevičius - why not expand on that a little and convert it to an answer? Commented Sep 5, 2018 at 21:27
• You will need a quite smart validation strategy to compare methods. Commented Sep 2, 2022 at 10:16