7
$\begingroup$

I have a situation in which I have $n$ observations, each with $p$ independent variables and $q$ dependent variables. I would like to build a model or series of models to obtain predictions of the $q$ dependent variables for a new observation.

One way is to build multiple models, each one predicting a single dependent variable. An alternative approach is to build a single model to predict all the dependent variables at one go (multivariate regression or PLS etc).

My question is: does taking into account multiple DV's simultaneously lead to a more robust/accurate/reliable model? Given the fact that some of the $q$ dependent variables might be correlated with each other, does this fact hamper or help a single model approach? Are there references that I could look up on this topic?

$\endgroup$
  • 2
    $\begingroup$ My experience is that fitting q correlated DVs together in one model can obtain more accurate estimates than fitting them separately. It is easy to distinguish the difference by simulations. $\endgroup$ – cchien Nov 9 '11 at 20:44
  • $\begingroup$ From what I was taught in school, one should build a model based on theoretical knowledge; that would prevent the analyst from fishing for results that just happen by chance (which happens a lot). Therefore, I would suggest that you base your model on some theoretical relationships from literatures and then expand from there. $\endgroup$ – King Nov 10 '11 at 10:49
5
$\begingroup$

You need to check for correlations amongst your dependent variables (edit: @BilalBarakat's answer is right, the residuals are what's important here). If all or some are independent, you can run separate analyses on each. If they are not independent, or whichever ones aren't, you could run a multivariate analysis. This will maximize your power while holding the type I error rate at your alpha level.

You should know, however, that this will not make your analysis more accurate/robust. This is a different issue than simply whether or not your model predicts the data better than the null model. In fact, with so much going on, unless you have a lot of data, it is likely that you could get very different parameter estimates with a new sample. It is even possible that the sign on a beta will flip. Much depends on the size of p and q and the nature of their correlation matrices, but the volume of data required for robustness can be massive. Remember that, although many people use 'significant' and 'reliable' as synonyms, they actually aren't. It is one thing to know that a variable is not independent of another variable, but another thing entirely to specify the nature of that relationship in your sample as it is in the population. It can be easy to run a study twice and find a predictor significant both times, but with the parameter estimate sufficiently different to be theoretically meaningful.

Furthermore, unless you are doing structural equation modeling, you can't very well incorporate your theoretical knowledge regarding the variables. That is, techniques like MANOVA tend to be rawly empirical.

Another approach is to utilize what you know about the issue at hand. For example, if you have several different measures of the same construct (you could check this with a factor analysis), you can combine them. This can be done by turning them into z scores and averaging them. Knowledge of other sources of correlation (e.g., common cause or mediation) could also be utilized. Some people are uncomfortable with putting so much weight on domain knowledge, and I acknowledge that this is a philosophical issue, but I think it can be a mistake to require the analyses to do all of the work and assume that this is the best answer.

As for a reference, any good multivariate textbook should discuss these issues. Tabachnick and Fidell is well regarded as a simple and applied treatment of this topic.

$\endgroup$
4
$\begingroup$

To contradict @gung's first paragraph (sorry!), you should actually check for correlations among the residuals in your multiple models, rather than for correlations among the dependent variables as such. The fact that the latter are correlated by itself tells you nothing about whether your estimates will improve by modelling them jointly.

$\endgroup$
  • $\begingroup$ This is right. Two dv's could be otherwise independent, but both influenced by the iv's. As a result, they would appear correlated in the raw data, but the residuals would not be, and that's what's more important. Good catch. $\endgroup$ – gung - Reinstate Monica Nov 13 '11 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.