Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Is it possible to have a (multiple) regression equation with two or more dependent variables? Sure, you could run two separate regression equations, one for each DV, but that doesn't seem like it would capture any relationship between the two DVs?

share|improve this question
add comment

6 Answers 6

up vote 15 down vote accepted

Yes, it is possible. What you're interested is is called "Multivariate Multiple Regression" or just "Multivariate Regression". I don't know what software you are using, but you can do this in R.

Here's a link that provides examples.

http://www.psych.yorku.ca/lab/psy6140/lectures/MultivariateRegression2x2.pdf

share|improve this answer
    
One might add that fitting the regressions separateley is indeed equivalent to the multivariate formulation with a matrix of dependent variables. In R with package mvtnorm installed (1st: multivariate model, 2nd: separate univariate models): library(mvtnorm); X <- rmvnorm(100, c(1, 2), matrix(c(4, 2, 2, 3), ncol=2)); Y <- X %*% matrix(1:4, ncol=2) + rmvnorm(100, c(0, 0), diag(c(20, 30))); lm(Y ~ X[ , 1] + X[ , 2]); lm(Y[ , 1] ~ X[ , 1] + X[ , 2]); lm(Y[ , 2] ~ X[ , 1] + X[ , 2]) –  caracal Nov 14 '10 at 12:26
    
Thanks, this is what I was looking for! There doesn't seem to be an option for this in SPSS, though I feel like I should be able to determine the coefficients of my equation from a multivariate GLM, no? –  Jeff Nov 14 '10 at 20:53
add comment

@Brett's response is fine.

If you are interested in describing your two-block structure, you could also use PLS regression. Basically, it is a regression framework which relies on the idea of building successive (orthogonal) linear combinations of the variables belonging to each block such that their covariance is maximal. Here we consider that one block $X$ contains explanatory variables, and the other block $Y$ responses variables, as shown below:

alt text

We seek "latent variables" who account for a maximum of information (in a linear fashion) included in the $X$ block while allowing to predict the $Y$ block with minimal error. The $u_j$ and $v_j$ are the loadings (i.e., linear combinations) associated to each dimension. The optimization criteria reads

$$ \max_{\mid u_h\mid =1,\mid v_h\mid =1}\text{cov}(X_{h-1}u_h,Yv_h)\quad \big(\equiv \max\text{cov}(\xi_h,\omega_h)\big) $$

where $X_{h-1}$ stands for the deflated (i.e., residualized) $X$ block, after the $h^\text{th}$ regression.

The correlation between factorial scores on the first dimension ($\xi_1$ and $\omega_1$) reflects the magnitude of the $X$-$Y$ link.

share|improve this answer
add comment

I would do this by first transforming the regression variables to PCA calculated variables, and then I would to the regression with the PCA calculated variables. Of course I would store the eigenvectors to be able to calculate the corresponding pca values when I have a new instance I wanna classify.

share|improve this answer
1  
This seems conceptually different than the answer above. I'm still not clear as to how transforming my variables to PCA coefficients allows me to regress on 2+ dependent variables? –  Jeff Nov 14 '10 at 20:55
    
@Jeff this answer is actually conceptually similar to multivariate regression. Here, the suggestion is to do two discrete steps in sequence (i.e., find weighted linear composite variables then regress them); multivariate regression performs the two steps simultaneously. Multivariate regression will be more powerful, as the WLCV's are formed so as to maximize the regression. However, the two-step procedure may provide more clarity regarding the process, or be otherwise preferable for the researcher. –  gung Jan 4 '12 at 22:17
add comment

Did you already come across the term "canonical correlation"? There you have sets of variables on the independent as well as on the dependent side. But maybe there are more modern concepts available, the descriptions I have are all of the eighties/nineties...

share|improve this answer
    
Canonical correlation is the correlation between factor scores computed from two-block structures, as with CCA or PLS. This is exactly what I described in my response (PLS regression), although PLS is more appropriate than CCA when the variables play an asymmetrical role, which is likely to be the case here. This is because there's an asymmetric deflation process and we work with the covariance instead (with CCA, we deflate both blocks at the same time, and we seek to maximize the correlation, instead of the covariance). –  chl Nov 16 '10 at 7:50
    
@chl: upps- today(end of january) I came back to this question/conversation of mid-november.... Sorry I didn't check earlier - there was something with my courses and then I forgot the stat.exchange... If I've something worth I'll come back next days. –  Gottfried Helms Jan 28 '11 at 23:23
add comment

Multivariate regression is done in SPSS using the GLM-multivariate option.

Put all your outcomes (DVs) into the outcomes box, but all your continuous predictors into the covariates box. You don't need anything in the factors box. Look at the multivariate tests. The univariate tests will be the same as separate multiple regressions.

As someone else said, you can also specify this as a structural equation model, but the tests are the same.

(Interestingly, well, I think it's interesting, there's a bit of a UK-US difference on this. In the UK, multiple regression is not usually considered a multivariate technique, hence multivariate regression is only multivariate when you have multiple outcomes / DVs.)

share|improve this answer
    
an addition at @Jeremy Miles answer: www-01.ibm.com/support/docview.wss?uid=swg21476743 –  Epaminondas Sep 26 '13 at 16:39
add comment

It's called structural equation model or simultaneous equation model.

share|improve this answer
2  
I could be wrong, but I don't think this is the same thing. From SEM graphs that I've seen, it looks like SEM uses multiple regression equations to determine the values of latent factors, and then another regression is run on the value of those latent factors to determine a higher-order factor. Maybe this is wrong, but I've never seen an SEM graph that links several IVs to multiple DVs-- everything is hierarchical. –  Jeff Nov 14 '10 at 20:59
    
Figure 8 in this paper: biomedcentral.com/1471-2288/3/27 You can do it, but there's little point. It's the same as MANOVA. –  Jeremy Miles Mar 25 '13 at 16:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.