# p-value from a binomial model glm for a binomial predictor

I'm trying to fit a glm to some data which are highly correlated (I can see this from the data). However, when I fit the glm it is giving a p-value of almost 1, which seems to indicate I'm not using the right test or have made a mistake. Does anyone know where I'm going wrong here? I've provided some example data that are perfectly correlated to illustrate my point. Example data/code:

x <- c(0,0,0,0,1,1,1,0,0,1,0,1,0,1,0,1,0,0,1,1,0,0,0,1,0,0,1,0,1)
y <- c(0,0,0,0,1,1,1,0,0,1,0,1,0,1,0,1,0,0,1,1,0,0,0,1,0,0,1,0,1)
fit <- glm(y ~ x, family = binomial('logit'))
summary(fit)


The problem is that the logistic regression has fitted values near to 0 and 1, and the asymptotic formula for standard errors in a binary regression are not at all accurate in this situation. The regression itself is fine, it just means you have to use a likelihood ratio test instead of a z-test to test significance:

> x <- c(0,0,0,0,1,1,1,0,0,1,0,1,0,1,0,1,0,0,1,1,0,0,0,1,0,0,1,0,1)
> y <- c(0,0,0,0,1,1,1,0,0,1,0,1,0,1,0,1,0,0,1,1,0,0,0,1,0,0,1,0,1)
> fit <- glm(y ~ x, family = binomial('logit'))
> anova(fit, test="Chi")
Analysis of Deviance Table

Model: binomial, link: logit

Response: y

Terms added sequentially (first to last)

Df Deviance Resid. Df Resid. Dev  Pr(>Chi)
NULL                    28     39.336
x     1   39.336        27      0.000 3.568e-10 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


As you can see, the p-value for the regression of $y$ on $x$ is $3.6\times 10^{-10}$. Highly significant!

This problem occurs in logistic regression whenever the fit is "too good" and the regression coefficient becomes infinitely large. In this case, dividing the coefficient by its standard error to get a z-statistic becomes meaningless (infinity divided by infinity) so you have to switch to the much better likelihood ratio test provided by anova.

Depending on the number of covariates you have ($p$), I suggest you try one of the following:

1. Use LASSO regression to decrease $p$.

2. Run PCA to get a hunch of the actual dimension of your data, or use SVD decomposition to see which singular values are close to 0.

3. Look for the condition number of your covariates matrix in order to find collinearity.

4. Examine a no-intersect model (R: glm(y ~ x -1, family = binomial('logit'))), especially if you have only one covariate.

Once you get a sense of the problem, a proper solution can be found. Personally I refrain from using Logistic Regression when $p=1$ as there are better classifiers (such as LDA).

• Hi! Thanks for your answer. I'm not sure I really understand though, the problem isn't that I need to decrease p per se. The problem is that at the moment the result for the example data is showing that x has no effect on y, when in fact y can be predicted perfectly if you have x (i.e. if x = 1, y = 1 too). Given that, it seems to me that I am doing something wrong! Commented Nov 6, 2017 at 9:50
• This IS NOT what it says. The noted p-value the the significance minimal level in which we'll reject the null hypothesis $H_0:\hat{\beta}_x=0$, under the model assumptions of Logistic Regression. If you only have vectors $y$ and $x$ and their correlation is high, there is no need for Logistic Regression. Commented Nov 6, 2017 at 10:03
• I must really be missing the point here, sorry for being slow. The reason for doing Logistic regression is that there are three predictors in the model (the other two are continuous predictors). The model described is similar to that in this answer: stats.stackexchange.com/a/311580/52956 Commented Nov 6, 2017 at 10:24
• Okay, so the model you use is nothing like the example you originally gave. Please post a screenshot of the glm summary or copy it, so I can see what's going on. Commented Nov 6, 2017 at 10:29
• OK, I have edited the output from the glmm summary in the original question. Thanks for the help! (presencet0 is the binomial predictor) Commented Nov 6, 2017 at 10:33