# Difference between several univariate regressions and one multivariate regression in a machine learning context

I know there are several other questions asking for the advantages of multivariate regression over several univariate regressions (e.g. this). I understand that the dependent variables can be correlated and that multivariate regression can account for that.

What I couldn't find, is whether this difference actually plays a role if all I want is to minimize the error (say Euclidean distance) on a particular test set after training. From my understanding, multivariate linear regression can hardly be more general than several univariate linear regressions.

Take a data set of examples where each example is a tuple $$$$ consisting of $$input \in \mathbb{R}^n$$ and $$target \in \mathbb{R}^m$$. Assume this data set is separated into two disjunct sets: the training set $$Train$$ and the test set $$Test$$.

Can you give me an artificial data set, where a multivariate linear (degree 1) regression for $$n$$ independent variables and $$m$$ dependent variables performs better than $$m$$ univariate linear (degree 1) regressions, each of which for $$n$$ independent variables and only one of the dependent variables? Both procedures are supposed to be generated from a random subset $$Train$$ consisting of 80% of all examples. Their success is supposed to be determined by the cumulative error over the disjunct subset $$Test$$ containing the remaining 20% of all examples.

I know that data sets can be constructed to favor any learning procedure over another. Exactly such a data set would help me understand the advantages of multivariate regression from a machine learning perspective.

• According to this page that was linked in the answer to the quetsion you referenced: stats.idre.ucla.edu/stata/dae/multivariate-regression-analysis "Separate OLS Regressions ...The individual coefficients, as well as their standard errors will be the same as those produced by the multivariate regression." – Jonny Lomond Apr 24 at 12:51