("Multivariate" regression in this post means a multidimensional response variable.)
I have been playing with multivariate regression for the past two days, and I have noticed something that did not make such an impression on me the first time I studied multivariate regression: it seems to be exactly the same as doing one regression per response variable.
If I do the multivariate regression, I get a matrix of parameters. If I do one regression per response variable, I get several vectors of parameters that can be concatenated to form the matrix of parameters, so doing the multivariate regression does not affect the parameter estimates.
Okay, the parameter estimates are the same either way. The next place where there could be differences is in standard errors on the parameters. The Rencher and Christensen book I'm consulting does not give a way to calculate standard errors for any individual value in the parameter matrix, but it gives a way to determine if a row of the parameter matrix is significant, which is pretty close and at least analogous to an F-test on one parameter in a vanilla OLS regression. However...
...nothing in that calculation seems to use the dependence of the response variable!
What's going on? What does multivariate regression get us that one regression per response variable lacks?
(For context, a lot of what I do is two-sample ANCOVA where I want to extend a two-sample t-test to account for sources of variability: the covariates in the regression. I now want to try this with multivariate response variables and test the indicator variable, I suppose a two-sample MANCOVA. But it seems that I do not get any advantage by doing one MANCOVA instead of one ANCOVA per response variable.)
Reference
Rencher, Alvin C., and William E. Christensen. Methods of Multivariate Analysis. 3rd ed., Wiley, 2012.
