2
$\begingroup$

I have two predictor variables: An indicator variable A and a continuous variable B. My response variable is continuous (and also bounded, have not made it logit for reasons of simplicity).

In simple linear regression, both predictors are significant. When including the two in multiple regression, both become insignificant in an overall significant model.

Other details:

  • An interaction variable composed of the two variables is insignificant
  • Beta for quantitative variable B is quite different between the two groups of the indicator variable A if I do a stratified analysis with a regression with B (That is, separate the data into the 0 and 1 groups for indicator A, then run regression using B as predictor).
  • VIF's are low for the two variables, suggesting no multicolinearity.

Crucial to the theory my study is about is the possibility that the variance explained by B is in large part explained by variable A. It doesn't seem like I am able to justifiably make any such claim.

But what can I say in regards to the impact of A and B on the response and their relationship with one another?

I am happy to provide more details if needed.


Requested info about confidence intervals:

Here are the intervals --

FAB.Average is quantitative predictor, the HSSorSTEM is the indicator

FAB.Average is quantitative predictor, the HSSorSTEM is the indicator. My n = 30, but each datapoint is the average of 500+ observations, if that makes any difference.

I'm not quite sure what a joint confidence ellipse is (I work with a knowledgable statistician, but am not myself one), but here was my try at making one:

enter image description here

$\endgroup$
4
  • $\begingroup$ Can your plot the joint confidence ellipse for the two coefficients? $\endgroup$ – Dimitriy V. Masterov Jul 15 '16 at 6:50
  • $\begingroup$ @DimitriyV.Masterov -- I tried to fulfill your request as best I can. $\endgroup$ – philosonista Jul 15 '16 at 8:31
  • $\begingroup$ This certainly looks like @Björn's second story is plausible (see here on why). Based on your other added info, I might consider a model where the you have both A, B, and their interaction (if that's not what you are doing already). I would also encourage you to use analytic weights that are inversely proportional to the variance of an observation. This can be helpful if observations represent averages and the weights are the number of elements that gave rise to the average. $\endgroup$ – Dimitriy V. Masterov Jul 15 '16 at 16:47
  • $\begingroup$ I think if you plot y versus B with the different marker symbols for A where the markers depend on size of the group, you should cleanly see if your story makes sense. $\endgroup$ – Dimitriy V. Masterov Jul 15 '16 at 16:48
2
$\begingroup$

Per-se the information that something goes from being significant to not being significant in a slightly different analysis does tell you very little. It happens all the time and may very often indicate that results were not very robust and compelling in the first place. This should usually not be overinterpreted (or as Gelman says:'The Difference between "Significant" and "Not Significant" Is Not Itself Statistically'). Particularly, if there is no massive change in the regression coefficients, my first suspicion would be that there is nothing very interesting going on here.

Other possibilities include that the two variables could be correlated and as a result the model has trouble deciding between whether to explain things through one variable or the other. In the end, it compromises thereby attenuating the regression coefficients for both variables (and the evidence in favor of an effect) and partially as a result of that the resulting p-values are not significant. In contrast, you could even get a situation, where one of the regression coefficients reverses sign and is significant with a coefficient that on the face of it appears to not make sense (i.e. if you do not condition on the knowledge about the other predictor), but from what you say that is not the case here.

$\endgroup$
1
  • $\begingroup$ The beta's do actually change quite a bit: From -16 not adjusting for the indicator variable to -9 when adjusting for the indicator variable. The two variables are indeed correlated: cor() gives -0.6. $\endgroup$ – philosonista Jul 15 '16 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.