7
$\begingroup$

I am getting astronomically high p-values for all the coefficients in a logistic regression model. I am not sure why they are so high:

  • 0.7096
  • 0.4441
  • 0.9783
  • 0.7826
  • 0.6890
  • 0.7538
  • 0.8017
  • 0.9332
  • 0.9564

I honestly cannot find out why they are so high. My dependent variable consists of ratios between 0 and 1, and my independent variables vary between continuous data, ordinal data, and categorical data.

Just adding a bit more information. I am creating a model to predict the proportion of those who evacuated to the remaining population in an area. For example, 10 percent evacuated / 90 percent left, and so on, over 37 hours. The data is sequential where each hour before effects the value of each hour after it. In the literature they call it the sequential logit model. Currently I am using Matlab's generalized linear model function using a binomial distribution linked with a logit. I am using 9 independent (predictor) variables and one dependent variable that is continuous between 0 and 1.

More information! The model being developed operates only on people that have already made the decision to evacuate. So I am not testing for if they will leave or not but when they will leave. Hopefully this clarifies what I am trying to do.

I downloaded R and I ran a logistic test. Here is what I got

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-0.41146  -0.12817  -0.00301   0.10222   0.47937  

Coefficients:
                   Estimate Std. Error z value Pr(>|z|)
(Intercept)       -36.87241  570.55243  -0.065    0.948
Hour               -0.27353    1.00636  -0.272    0.786
LN.TU.             -3.96075    8.57513  -0.462    0.644
as.factor(Evac1)1   3.34697   13.38238   0.250    0.803
as.factor(Evac2)1   1.42544    3.51690   0.405    0.685
Pressure            0.04122    0.59701   0.069    0.945
WindSpeed          -0.01583    0.23526  -0.067    0.946
as.factor(Time1)1  10.11888   11.22222   0.902    0.367
as.factor(Time2)1   9.89199   10.45510   0.946    0.344
as.factor(Time3)1  10.72968    9.48993   1.131    0.258

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 14.5671  on 36  degrees of freedom
Residual deviance:  1.5175  on 27  degrees of freedom
AIC: 27.927

Number of Fisher Scoring iterations: 9

My p-values remain very unappealing. I am beginning to think that instead of trying to fix this, I should just believe that what I am doing is wrong and take another approach. Thanks all who answered and helped!

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ What is your sample size? In small samples, Wald tests are conservative in logistic regression. Also, how many independent variables do you have? $\endgroup$
    – guest
    Commented Mar 1, 2012 at 4:11
  • 3
    $\begingroup$ In addition to what guest said, on a more basic level - are you sure these variables are associated with the dependent variable? Also, is the predictors correlated with each other? $\endgroup$
    – Macro
    Commented Mar 1, 2012 at 4:14
  • $\begingroup$ Your regression model contains 9 predictor variables but you only have 37 observations? This website about overfitting might give some insights into your problem. $\endgroup$
    – Bernd Weiss
    Commented Mar 1, 2012 at 13:32
  • 2
    $\begingroup$ Are you sure your model can handle percentages? Usual logistic regression requires the response be a count of successes out of a known number of trials. So to fit this data with a standard logistic regression you'd have to tell it how many trials there were, and p-values can vary wildly based on how many trials you have, even if the percentages are all the same. $\endgroup$
    – Aaron - mostly inactive
    Commented Mar 1, 2012 at 14:44
  • 3
    $\begingroup$ Normally, a logit model of any type has a strictly binary (0-1) dependent variable. How are you even managing to carry out the calculations with a "continuous" variable? It's possible that Matlab is just cranking through completely meaningless operations. $\endgroup$
    – whuber ♦
    Commented Mar 1, 2012 at 15:47

3 Answers 3

Reset to default
14
$\begingroup$

There are many possible explanations but some of the most common are:

  1. The explanatory variables are not related to your response. So no problem here, you just have a negative finding.

  2. Some or all of the variables are related, but they are highly correlated so you have a problem with "variance inflation factors". If you removed some of the variables you might find that the remaining ones do seem to make a significant contribution. However, this is not a solution for finding a final model - search for "multicollinearity" to find some of the issues and possible solutions.

  3. Your sample size is too small.

  4. Your model is mis-specified in some other way eg the continuous data actually have a non-linear relationship to the logit of your response.

  5. Your data is under- or over-dispersed for some reason, more than would be expected of a binomial variable (in which case you might be able to fix the problem by fitting a quasi likelihood model).

  6. You somehow scrambled your data during manipulating it into shape to fit your model.

There's probably more but those are the obvious ones.

Improve this answer
$\endgroup$
8
  • 5
    $\begingroup$ 1. 9/10 null p-values above 0.5 is suspicious. 2. Multicollinearity doesn't bias results towards insignificance 3. Too small for what? 4. Non-linearity? Under the null, the relationship is flat, and that's what we care about here. 5. It is impossible for binary random variables to be over-dispersed, and they are the default in logistic regression - not "binomial" regression. $\endgroup$
    – guest
    Commented Mar 1, 2012 at 6:39
  • $\begingroup$ Guest, re: 4-- For example, the best linear estimate of a $y = x^2$ when $x$ is centered around 0 is a flat line. I think this is what Peter Ellis wsz getting at. re: 5-- this is absolutely right and I'm glad you pointed it out - it's a common misunderstanding. Also, re: 3-- Peter probably was referring to a lack of power. $\endgroup$
    – Macro
    Commented Mar 1, 2012 at 12:21
  • $\begingroup$ re 5, what about jstor.org/pss/1390627 (genuine question, this isn't really my field of expertise)? But I was mainly thinking about under dispersion which, unless my brain is backwards, would mean that the reported standard errors are larger than they should be if the under dispersion were taken into account. The people deciding to evacuate are not all taking iid decisions, they are influenced by eachother. $\endgroup$
    – Peter Ellis
    Commented Mar 1, 2012 at 18:56
  • $\begingroup$ you can model an overdispersed ${\rm Binomial}(n,p)$ variable as long as $n > 1$. There is no flexibility in a bernoulli trial in terms of inflating the variance while leaving the mean fixed. $\endgroup$
    – Macro
    Commented Mar 1, 2012 at 19:33
  • 1
    $\begingroup$ @PeterEllis, no, it doesn't matter if there are random variables in the mean. If a binary outcome has mean p (which you may obtain by averaging over some other random variables) then it has variance p(1-p). There is only one distribution possible; Y takes the value 1 with probability p and 0 with probability 1-p. Overdispersion relative to specified models is a real enough issue in other situations, but with binary outcomes it is not. $\endgroup$
    – guest
    Commented Mar 9, 2012 at 20:54
4
$\begingroup$

Because your response is a ratio between 0 and 1, it might be possible there is a heterogeneous (skewness) between data. In this case, logistic regression can not take into account this heterogeneity very well. An alternative model is beta regression. Beta distribution is a good choice to consider heterogeneity and skewness between data. Also, there is a R package to do this regression.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Interesting point, I will definitely try the beta distribution. Thanks! $\endgroup$
    – shiu6rewgu
    Commented Mar 1, 2012 at 12:13
  • 1
    $\begingroup$ If you are looking for when they are decidng to leave, then you probably want a survival analysis of some kind And if everyone has left, then what are the proportions? The questions you are asking, the data you have, and the methods you are running don't match up. $\endgroup$
    – Peter Flom
    Commented Mar 2, 2012 at 1:09
  • $\begingroup$ As @Peter Flom has noticed, if you want to model the time of leaving, your problem would be a survival modeling. On the other hand, if a logistic regression is what you want to do, your responses are (dynamically) correlated, as I grasped in new edited description, and you have to use a dynamic logistic regression. $\endgroup$
    – hbaghishani
    Commented Mar 2, 2012 at 6:16
2
$\begingroup$

I've seen p-values very close to 1.0 in ordinary least squares. A likely explanation is omitting an important explanatory variable. That inflates the error term making other variables look "suspiciously insignificant". A quickly constructed simulation follows but I've seen real cases that were more dramatic.

Omitted Included Omitted + in Model N(0,1)/10 1 1 1.21268754 0 1 0.22398027 1 1 1.15664332 0 1 -0.0321136 1 1 0.84352449 0 1 -0.0770372 1 1 1.01447926 0 1 0.00845824 1 1 0.97894666 0 1 -0.160745 1 2 0.91627849 0 2 0.04907199 1 2 1.05989843 0 2 0.09999067 1 2 0.86461229 0 2 -0.0017369 1 2 0.9341983 0 2 0.03245005 1 2 1.09566388 0 2 0.22768268

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.