0
$\begingroup$

I performed a logistic regression on my data where the dependent binary variable $Y$ have $0$ & $1$ values and the independent variables $X$ being binary as well as continuous. The regression results suggest that none of my three $X$ variables have $p<0.05$ which is very disappointing as I strongly believed them to be affecting the outcome of $Y$. Stratifying the data makes no difference as well. I can't help but think there might be some problem with my data. So I noticed that only $13\%$ of the observations $(n = 4080)$ actually have $1$ as $Y$ value. So a huge majority have $0$ and maybe this is causing the regression to show no statistically significant relation? Also all $X$ variables are different from each other so there is no multi colinearity. I'm new to statistics so I just want to make sure the data is not the issue here. Or is it that I was wrong all along and there might not be a relation after all. Any thoughts?

$\endgroup$

2 Answers 2

1
$\begingroup$

You're right, you should be slightly concerned about the power of your analysis. In general, 50% 1s and 0s will lead to more power.. but that's only half the story. 400 successes and 3600 failures is typically a LOT of power, depending on the distribution of the $X$ values.

Cross tabulate values of each $X$ variable and $Y$ as well (assuming $X$ is categorical as well). ``A chain is only as strong as its weakest link'' comes to mind: the power of the logistic regression model to detect an association will generally be greatly limited by the smallest cell count in the model.

This is because for a log odds ratio (natural parameter for a logistic regression model, in the $2 \times 2$ contingency table for outcomes:

$$ \begin{array}{c|cc} & Y & \bar{Y} \\ \hline X & A & B \\ \bar{X} & C& D \\ \end{array}$$

the log odds ratio $LOR = \log \frac{AD}{BC}$ has standard error:

$\mbox{se} ( LOR)^2 = 1/ A + 1/B + 1/C + 1/D$

So, the larger $A, B, C$ and $D$ are, the closer those fractions are to 0, and the mutual se is closer to 0.

$\endgroup$
4
  • $\begingroup$ I think you are correct, crosstabs doesn't show any statistical significance as well. 70% of observations with Y value 0 have X value 0 but also same 70% of observations with Y value 1 have X value 0. So no relation I guess. $\endgroup$
    – Salman
    Aug 3, 2015 at 21:41
  • $\begingroup$ @Salman are there 0 counts in these cells? Or counts of, say, less than 10? $\endgroup$
    – AdamO
    Aug 3, 2015 at 21:49
  • $\begingroup$ No there are no such small counts. The minimum count is 127 $\endgroup$
    – Salman
    Aug 3, 2015 at 22:14
  • $\begingroup$ @Salman then you should have high power. That means the results strongly suggest there is no association, unless you've run your model incorrectly. $\endgroup$
    – AdamO
    Aug 3, 2015 at 22:23
0
$\begingroup$

From my understanding of Logistic regression, you want to check that each category of yes/no's or 1/0's has a count >10*(p-1), where p is the number of covariates + 1 (for the intercept). If this holds true, you should be good as far as your dependent variables go. There is good reasoning to have closer to a 50/50 ratio of 1/0's but you don't have to.

Have you considered checking for Goodness of Fit using a Hosmer-Lemeshow test Statistic to see if logistic regression is appropriate with your covariates? To do in R, you can use this code

library(ResourceSelection) 
hoslem.test()$p.value #look up the parameters in R's help file

If you aren't using R, I'm not sure how to do this test.

$\endgroup$
2
  • $\begingroup$ Hmmm, my data does not have the 50/50 distribution but it satisfies the condition you mentioned easily. Yes Hosmer Lemershow test also shows a 0.129 which is a good fit i guess? But it doesn't change the fact that there is still no statistical significance shown by logistic regression. $\endgroup$
    – Salman
    Aug 3, 2015 at 21:44
  • $\begingroup$ P value > than alpha means there is no evidence against the usefulness of logistic regression, i.e. your data passes the Homser-Lemeshow test. As far as your covariates go, how are you checking for statistical significance? $\endgroup$
    – Jake
    Aug 4, 2015 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.