# Why are my p-values so high?

I am getting astronomically high p-values for all the coefficients in a logistic regression model. I am not sure why they are so high:

• 0.7096
• 0.4441
• 0.9783
• 0.7826
• 0.6890
• 0.7538
• 0.8017
• 0.9332
• 0.9564

I honestly cannot find out why they are so high. My dependent variable consists of ratios between 0 and 1, and my independent variables vary between continuous data, ordinal data, and categorical data.

Just adding a bit more information. I am creating a model to predict the proportion of those who evacuated to the remaining population in an area. For example, 10 percent evacuated / 90 percent left, and so on, over 37 hours. The data is sequential where each hour before effects the value of each hour after it. In the literature they call it the sequential logit model. Currently I am using Matlab's generalized linear model function using a binomial distribution linked with a logit. I am using 9 independent (predictor) variables and one dependent variable that is continuous between 0 and 1.

More information! The model being developed operates only on people that have already made the decision to evacuate. So I am not testing for if they will leave or not but when they will leave. Hopefully this clarifies what I am trying to do.

I downloaded R and I ran a logistic test. Here is what I got

Deviance Residuals:
Min        1Q    Median        3Q       Max
-0.41146  -0.12817  -0.00301   0.10222   0.47937

Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)       -36.87241  570.55243  -0.065    0.948
Hour               -0.27353    1.00636  -0.272    0.786
LN.TU.             -3.96075    8.57513  -0.462    0.644
as.factor(Evac1)1   3.34697   13.38238   0.250    0.803
as.factor(Evac2)1   1.42544    3.51690   0.405    0.685
Pressure            0.04122    0.59701   0.069    0.945
WindSpeed          -0.01583    0.23526  -0.067    0.946
as.factor(Time1)1  10.11888   11.22222   0.902    0.367
as.factor(Time2)1   9.89199   10.45510   0.946    0.344
as.factor(Time3)1  10.72968    9.48993   1.131    0.258

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 14.5671  on 36  degrees of freedom
Residual deviance:  1.5175  on 27  degrees of freedom
AIC: 27.927

Number of Fisher Scoring iterations: 9


My p-values remain very unappealing. I am beginning to think that instead of trying to fix this, I should just believe that what I am doing is wrong and take another approach. Thanks all who answered and helped!

-
What is your sample size? In small samples, Wald tests are conservative in logistic regression. Also, how many independent variables do you have? –  guest Mar 1 '12 at 4:11
In addition to what guest said, on a more basic level - are you sure these variables are associated with the dependent variable? Also, is the predictors correlated with each other? –  Macro Mar 1 '12 at 4:14
I have a sample size of 37 and I am using 9 independent variables. To be honest I am using independent variables that have been established in literature over the past 10-15 years so at first I just assumed they were significant. But when I started doing the tests and started to get these wild p-values I wasn't sure whether I was doing something wrong or the precedence was wrong. I have yet to check the associations between the variables and the dependent variable. Once I do that I will report back here. I am also pretty sure that the predicators do not really correlate with one another.Thanks –  tenki Mar 1 '12 at 12:11
Are you sure your model can handle percentages? Usual logistic regression requires the response be a count of successes out of a known number of trials. So to fit this data with a standard logistic regression you'd have to tell it how many trials there were, and p-values can vary wildly based on how many trials you have, even if the percentages are all the same. –  Aaron Mar 1 '12 at 14:44
Normally, a logit model of any type has a strictly binary (0-1) dependent variable. How are you even managing to carry out the calculations with a "continuous" variable? It's possible that Matlab is just cranking through completely meaningless operations. –  whuber Mar 1 '12 at 15:47
show 3 more comments

There are many possible explanations but some of the most common are:

1. The explanatory variables are not related to your response. So no problem here, you just have a negative finding.

2. Some or all of the variables are related, but they are highly correlated so you have a problem with "variance inflation factors". If you removed some of the variables you might find that the remaining ones do seem to make a significant contribution. However, this is not a solution for finding a final model - search for "multicollinearity" to find some of the issues and possible solutions.

3. Your sample size is too small.

4. Your model is mis-specified in some other way eg the continuous data actually have a non-linear relationship to the logit of your response.

5. Your data is under- or over-dispersed for some reason, more than would be expected of a binomial variable (in which case you might be able to fix the problem by fitting a quasi likelihood model).

6. You somehow scrambled your data during manipulating it into shape to fit your model.

There's probably more but those are the obvious ones.

-
1. 9/10 null p-values above 0.5 is suspicious. 2. Multicollinearity doesn't bias results towards insignificance 3. Too small for what? 4. Non-linearity? Under the null, the relationship is flat, and that's what we care about here. 5. It is impossible for binary random variables to be over-dispersed, and they are the default in logistic regression - not "binomial" regression. –  guest Mar 1 '12 at 6:39
Thank you for your responses Peter and guest. I will definitely look over the points you both made. –  tenki Mar 1 '12 at 12:12
Guest, re: 4-- For example, the best linear estimate of a $y = x^2$ when $x$ is centered around 0 is a flat line. I think this is what Peter Ellis wsz getting at. re: 5-- this is absolutely right and I'm glad you pointed it out - it's a common misunderstanding. Also, re: 3-- Peter probably was referring to a lack of power. –  Macro Mar 1 '12 at 12:21
re 5, what about jstor.org/pss/1390627 (genuine question, this isn't really my field of expertise)? But I was mainly thinking about under dispersion which, unless my brain is backwards, would mean that the reported standard errors are larger than they should be if the under dispersion were taken into account. The people deciding to evacuate are not all taking iid decisions, they are influenced by eachother. –  Peter Ellis Mar 1 '12 at 18:56
@PeterEllis, no, it doesn't matter if there are random variables in the mean. If a binary outcome has mean p (which you may obtain by averaging over some other random variables) then it has variance p(1-p). There is only one distribution possible; Y takes the value 1 with probability p and 0 with probability 1-p. Overdispersion relative to specified models is a real enough issue in other situations, but with binary outcomes it is not. –  guest Mar 9 '12 at 20:54
show 4 more comments

Because your response is a ratio between 0 and 1, it might be possible there is a heterogeneous (skewness) between data. In this case, logistic regression can not take into account this heterogeneity very well. An alternative model is beta regression. Beta distribution is a good choice to consider heterogeneity and skewness between data. Also, there is a R package to do this regression.

-
Interesting point, I will definitely try the beta distribution. Thanks! –  tenki Mar 1 '12 at 12:13
I appreciate your response but I cannot deviate from what has already been established in literature, at least for now. Is there anyway I can effectively analyze this data while using logistic regression? Because when I go over the journal articles I see that they are getting p-values that are significant at the .05 level while I am getting p-values that are around .9. Maybe I am doing something wrong I am not longer sure –  tenki Mar 2 '12 at 1:06
If you are looking for when they are decidng to leave, then you probably want a survival analysis of some kind And if everyone has left, then what are the proportions? The questions you are asking, the data you have, and the methods you are running don't match up. –  Peter Flom Mar 2 '12 at 1:09
Sorry about that. I have very little statistical knowledge other than what is taught in AP Stat (still taking the course now too) so this is all very new to me, especially logistic regression. I will try and think this through again then try and explain myself again. But yes I am trying to predict when people will evacuate when a hurricane touchdown is imminent. My independent variables are things like hurricane speed, time to landfall, time of day (binary), and evacuation notice(binary). I know that logistic regression was used before hand, and glm linked logit was the only one in matlab –  tenki Mar 2 '12 at 1:28
so I used it because of that –  tenki Mar 2 '12 at 1:28
show 2 more comments

I've seen p-values very close to 1.0 in ordinary least squares. A likely explanation is omitting an important explanatory variable. That inflates the error term making other variables look "suspiciously insignificant". A quickly constructed simulation follows but I've seen real cases that were more dramatic.

Omitted Included Omitted + in Model N(0,1)/10 1 1 1.21268754 0 1 0.22398027 1 1 1.15664332 0 1 -0.0321136 1 1 0.84352449 0 1 -0.0770372 1 1 1.01447926 0 1 0.00845824 1 1 0.97894666 0 1 -0.160745 1 2 0.91627849 0 2 0.04907199 1 2 1.05989843 0 2 0.09999067 1 2 0.86461229 0 2 -0.0017369 1 2 0.9341983 0 2 0.03245005 1 2 1.09566388 0 2 0.22768268

-
hmmm, interesting point. Fact is there are tons of explanatory variables that could be causing the data to change as it does, especially in a hurricane evacuation situation, but data is very scarce in the world of hurricane evacuation. If this is the case that could explain a lot. Thank you very much with for your input. I also noticed that when I increase the sample size the p-values of some of the variables decrease to below .001 (with a sample size of 483) but that is only for 2-3, the others remain above 0.2. –  tenki Mar 7 '12 at 19:28