9
$\begingroup$

Would anyone be willing to venture an intuitive description of the situations under which a multivariate response model is more appropriate than many linear regressions?

As an example, take a randomly allocated agricultural extension program, and yields of several different crops grown by farmers. You could run several different models for each crop. Or you could aggregate the crops somehow. Or maybe you could run a multivariate response model, whereby your dependent variable is actually a matrix rather than a vector.

I've been reading up on the math of it all, but I haven't found a good intuitive description of the situations where these sorts of models are the most useful, nor their practical pitfalls. I get that the errors will be correlated between responses. Does this mean that you'd get more power in a situation where individual regressions would be underpowered? Is there any reason why coefficient matrices estimated in these models wouldn't have a causal interpretation if a variable is randomly allocated?

Improve this question
$\endgroup$
1
  • $\begingroup$ I've posted my comment and response from a moment ago` as an answer, but I am not sure whether the update answers your question or not. Can you take a look and maybe clarify your followup question if it doesn't? $\endgroup$
    – Glen_b
    Jun 7, 2013 at 7:03

1 Answer 1

Reset to default
8
$\begingroup$

I think my comments have grown long enough for an answer...

One reason why you might want to look at the multivariate case rather than univariate cases is when there's a lot of dependence between variables. It's quite possible for each univariate response to show "no effect" but the multivariate one to show a strong one. See this plot about a difference between two groups on just two dimensions

Note that here, $y$ and $x$ are both DVs, and the grouping variable (red/black indicator) is the (lone) IV in the 'regression'.

two groups, two dependent DVs

The issue is that the thing whose mean really differs between the two groups is not the variable $X$ or the variable $Y$ (that is, $\mu_{X2}-\mu_{X1}$ is almost zero, same for $Y$), but a particular linear combination - in the example, $Y-X$ - on which the means of the two groups strongly differ.

In that case univariate $t$ tests find nothing but a multivariate test sees it easily (which can be done by regression and multivariate regression where there is a single IV, the group indicator).

The same issue applies to other, less simple regressions.

Improve this answer
$\endgroup$
10
  • $\begingroup$ I think that makes sense. You use the correlations between DV's to be more certain about groupwise differences in independent variables (which can be generalized to more complicated models). But the coefficients themselves should be identical. $\endgroup$
    – generic_user
    Jun 7, 2013 at 7:20
  • $\begingroup$ But that leaves me with the question: are there any practical pitfalls? Difference in interpretation of coefficients that are important? I don't immediately see any... $\endgroup$
    – generic_user
    Jun 7, 2013 at 7:26
  • $\begingroup$ You still seem to have at least some confusion. You already know the IVs (the predictor) is different (if black is '0' and red is '1', you already know 0 and 1 differ - you chose them that way!). It's the response (the two axes in my plot are both response variables, DVs) and its those that you want to find effects for (in this simple example find differences in their means by regression on the 0-1 variable). It occurs to me that you may be alluding to seemingly unrelated regressions; is that the case? $\endgroup$
    – Glen_b
    Jun 7, 2013 at 7:31
  • $\begingroup$ I said independent, I meant dependent. $\endgroup$
    – generic_user
    Jun 7, 2013 at 7:33
  • 1
    $\begingroup$ They do have a bigger mean but the difference in population means very small compared to the size of the standard deviations in each variable. The average of the pair-differences is relatively much larger compared to the standard deviation by comparison. That is, there's a linear combination of the original variables on which there's a strong group difference effect even though the marginal effects are small. $\endgroup$
    – Glen_b
    Mar 2, 2019 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.