1
$\begingroup$

I've read a variety of answers on this topic and as far as I can see, the consensus is that multivariate regression is different from multiple, individual linear regressions. I also understand that in multivariate regression, the results are estimated simultaneously, which in principle could result in different outcomes.

However, I believe I am missing something in my understanding because I can't reproduce an example where they produce different results.

For example, using R syntax, 3 regressions with correlated errors

error <- rnorm(100,0,1)

x <- rnorm(100,0,1)

y <- rnorm(100,0,1)

z1 <- 1*x + 2*y + error

z2 <- 1.5*x + 0.5*y + error

z3 <- 0.8*x + 1*y + error

Result:

lm(cbind(z1, z2, z3) ~ x + y)

gives the exact same results as:

lm(z1 ~ x + y) lm(z2 ~ x + y) lm(z3 ~ x + y)

The results here are the same for both methods and I cant think of an example where the results wouldn't be the same.

Simply, could you give an example where multivariate regression produces different results to multiple, individual regressions?

$\endgroup$
1

1 Answer 1

0
$\begingroup$

I've read the related link noted above, particularly the comment regarding the UCLA example. I wanted to highlight a section as an answer in case someone else was struggling with the differentiation.

Quoted from the UCLA webpage: "You could analyze these data using separate OLS regression analyses for each outcome variable. The individual coefficients, as well as their standard errors will be the same as those produced by the multivariate regression. However, the OLS regressions will not produce multivariate results, nor will they allow for testing of coefficients across equations."

Therefore, it seems from an estimation perspective that the results are exactly the same. The difference appears only on whether you want to test the coefficients across the equations, against a hypothesis, simultaneously.

$\endgroup$
2
  • $\begingroup$ This reads more like a comment on the apparent duplicate thread. As a reply to the unique question you have asked--"could you give an example where multivariate regression produces different results"--it isn't an answer. Moreover, it isn't generally correct, because multivariate regression can give different results than the separate regressions. I suspect the page you are quoting is assuming the response variables are uncorrelated. It uses Stata's mvreg and the technical notes for its manual page confirm that. $\endgroup$
    – whuber
    Commented Feb 27, 2020 at 14:38
  • $\begingroup$ I'm happy to be corrected, if you could provide an example of where they would differ that would be very helpful. Given that the concrete example in the linked comment shows the same results, its quite confusing! $\endgroup$
    – user275010
    Commented Feb 27, 2020 at 17:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.