1
$\begingroup$

I have a binary response variable with 9 predictor variables. Lets denote the predictors $A, B, C, D, E...$

Suppose I run a model $y_i = \beta_0 + \beta_1 A + \beta_2 B + \beta_3 C + \cdots$. The results from the model tell me that $A$ and $C$ are the only significant covariates.

Now, my question is that is there any sense (or can insight be gained) from running individual models such as $$\begin{align} y_i &= \beta_0 + \beta_1 \cdot A \\ y_i &= \beta_0 + \beta_1 \cdot B \\y_i &= \beta_0 + \beta_1 \cdot C \\ \vdots \end{align}$$

Would I expect the individual covariate models to return the same significance as the combined covariate model?

If so, what information can running the individual models give me that I won't get from running the combined model.

side note: I've refrained from using the word multivariate to denote the model $y_i = \beta_0 + \beta_1 A + \beta_2 B + \beta_3 C + \cdots$. I've heard somewhere that the multivariate model actually represents multiple response variables. If I am wrong about this, please let me know in the comments.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
2
$\begingroup$

Would I expect the individual covariate models to return the same significance as the combined covariate model?

You mean the same p-values? No. Indeed even the sign of the coefficients could change.

If so, what information can running the individual models give me that I won't get from running the combined model.

Well, you'll have an estimate of the relationship when you don't take account of any other important predictors. In general I don't think there's much value in that, except perhaps to see how much additional variation the bigger model explains or how much the coefficients change when you do it. In some particular situations it may have some additional value.

side note: I've refrained from using the word multivariate to denote the model yi=β0+β1A+β2B+β3C+⋯. I've heard somewhere that the multivariate model actually represents multiple response variables.

Correct. With multiple covariates (/IVs) you have "multiple regression".

You didn't refrain from it in your title. Since it sounds like you meant to, I'll edit

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks. I don't think that the single covariate models give me information in my case. But, suppose I run a model with 9 predictors. Only 3 of them are significant. Can I run a new model with just those three predictors? Will this new model be a better suited model? $\endgroup$
    – masfenix
    Jul 30, 2015 at 4:06
  • 1
    $\begingroup$ There are problems with this strategy - parameter estimates in the new model are biased away from zero, standard errors are too small, p-values too small. It would be a good idea (if you can get your hands on it for a day or two) to read ch4 of Frank Harrell's Regression Modeling Strategies. This is not just a problem with p-value based model selection though, many kinds of model selection suffer similar issues $\endgroup$
    – Glen_b
    Jul 30, 2015 at 4:35
  • $\begingroup$ Also see here and here $\endgroup$
    – Glen_b
    Jul 30, 2015 at 4:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.