0
$\begingroup$

I am a little bit stuck with my data analysis.

What does it mean if your main effect is not significant (.442) but becomes even less significant when you add your control variable (.718) Furthermore, it seems that the contol variable has a marginal significant effect on the dependent variable (p=.104)

Many thanks in advance!

Improve this question
$\endgroup$
1
  • $\begingroup$ Assuming the analysis was conducted correctly, it seems like the main effect simply wasn't statistically significant. I don't think it really "means" much of anything -- neither the main effect nor the confounder was significantly associated with the response. Disappointing as that may be, it happens! Also, opinions differ on this point, but I tend to shy away from talking about "marginal significance" (especially with a p-value that far from the conventional .05 level). Report the p-value and report the effect size, and let the reader draw his/her own conclusions about the importance. $\endgroup$
    – dlid
    Commented Jun 21, 2018 at 18:46

2 Answers 2

Reset to default
1
$\begingroup$

If your control variable was measured prior to the predictor of interest, what you have observed usually occurs when the control variable has an effect in the same direction on both the predictor and the outcome. E.g., if prior academic achievement positively affects both admission into a scholarship program and future achievement, then the effect of the scholarship program on future achievement will be smaller conditional on prior achievement. In addition, if the control variable is highly associated with the predictor and less associated with the outcome, your increase in p-value may be due to colinearity or conditioning on a near-instrument, both of which don't change the effect of the predictor but increase it's variance.

If you control variable was measured after the predictor of interest, you may be blocking the effect of the predictor on the outcome. In the same scenario, imagine you instead control for an intermediate outcome, like depressed feelings. Part of the effect of the scholarship on future achievement may occur through the effect of the scholarship on lowering depressed feelings, which improve achievement. Controlling for depressed feelings will block part of this effect, decreasing the predictor's effect.

Improve this answer
$\endgroup$
-1
$\begingroup$

It means that your quantitative analysis may not yield the results, there could be an error, or the sample was not a good one.

Improve this answer
$\endgroup$
1
  • $\begingroup$ You can triangulate the analysis by using another method of analysis for comparison purposes $\endgroup$
    – Dr. Eldard Mukasa
    Commented Jun 22, 2018 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.