I know there are several other questions asking for the advantages of multivariate regression over several univariate regressions (e.g. this). I understand that the dependent variables can be correlated and that multivariate regression can account for that.
What I couldn't find, is whether this difference actually plays a role if all I want is to minimize the error (say Euclidean distance) on a particular test set after training. From my understanding, multivariate linear regression can hardly be more general than several univariate linear regressions.
Take a data set of examples where each example is a tuple $<input,~ target > $ consisting of $input \in \mathbb{R}^n $ and $target \in \mathbb{R}^m $. Assume this data set is separated into two disjunct sets: the training set $Train $ and the test set $Test $.
Can you give me an artificial data set, where a multivariate linear (degree 1) regression for $n $ independent variables and $m $ dependent variables performs better than $m $ univariate linear (degree 1) regressions, each of which for $n $ independent variables and only one of the dependent variables? Both procedures are supposed to be generated from a random subset $Train $ consisting of 80% of all examples. Their success is supposed to be determined by the cumulative error over the disjunct subset $Test $ containing the remaining 20% of all examples.
I know that data sets can be constructed to favor any learning procedure over another. Exactly such a data set would help me understand the advantages of multivariate regression from a machine learning perspective.
