2
$\begingroup$

I used the clogit function (from the survival package) to run a conditional logistic regression in R with a big dataset of 1:M matched pairs with n=300368964 and number of events= 39995.

model <- clogit(Alliance ~ OVB + CVC + BVB + strata(Strata), method="exact")    

I received following results:

                 coef  exp(coef)   se(coef)       z Pr(>|z|)    
OVB        -0.0498174  0.9514031  0.0166275  -2.996  0.00273 ** 
BVB         0.0277405  1.0281289  0.0304956   0.910  0.36300    
CVC         1.1709851  3.2251683  0.1089709  10.746  < 2e-16 ***
EarlyStage -1.3215824  0.2667129  0.0205851 -64.201  < 2e-16 ***
AvgVCSize   0.0087976  1.0088364  0.0002035  43.224  < 2e-16 ***
NumberVC    0.0643579  1.0664740  0.0034502  18.653  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Rsquare= 0   (max possible= 0.001 )
Likelihood ratio test= 6511  on 6 df,   p=0
Wald test            = 6471  on 6 df,   p=0
Score (logrank) test = 6801  on 6 df,   p=0

Since Rsquare equals 0 and the test ratios seems very high, I tried to plot the results to check whether the model fits. But I wasn't able to plot it properly.

I would online many papers which use the ratio Prob > chi2 = 0 from Stata as test ratio to proof the model fit.

How could I calculate this ratio in R? Are there any other ways I could check the model fit of my clogit results?

I would appreciate any help.

Thanks you very much in advance.

$\endgroup$
  • $\begingroup$ Your $n$ is a third of a billion observations and you have 40K events. It's not remotely surprising your p-values are all zero, even though the model doesn't explain a lot of the variation (& how is $R^2$ defined here?). I have no idea what you mean by "use the ratio Prob > chi2 = 0 ... as test ratio to proof the model fit". First, you don't 'prove' a model fit, and secondly, your initial expression is unclear. Prob-what is greater than $\chi^2$-what? Are you referring to the p-value for one of the model hypothesis tests? What about it? A low p-values doesn't imply the model is a good fit. $\endgroup$ – Glen_b Jan 25 '14 at 6:54
  • $\begingroup$ Sorry for the unclear expression. Prob is the probability of obtaining the chi-square statistic given that the null hypothesis is true. In other words, this is the probability of obtaining this chi-square statistic (6511) if there is in fact no effect of the independent variables, taken together, on the dependent variable. This is the p-value, which is compared to a critical value, perhaps .05 or .01 to determine if the overall model is statistically significant. $\endgroup$ – user37838 Jan 25 '14 at 13:27
  • $\begingroup$ If the p-value would be less than .000, I could say that the model is statistically significant. Is there maybe also a way to infer from the given model hypothesis tests whether the model could be significant? Or does the high number of observations compared to the events make it very unlikely that the model is statistically significant? $\endgroup$ – user37838 Jan 25 '14 at 13:50
  • $\begingroup$ p-values are conditional probabilities - they can't ever be less than zero. It sounds like your understanding of p-values (and statistical significance) is wrong. $\endgroup$ – Glen_b Jan 25 '14 at 14:24
  • $\begingroup$ With "less than .000", I meant that the p-value is very close to 0. I found the explanantion for the logit estimates here. $\endgroup$ – user37838 Jan 25 '14 at 14:44
0
$\begingroup$

In the diagram below, the parts on the right are from the document you linked, and the parts on the left are in your output in your question. I have marked corresponding parts with the same colors (the values won't be the same in this case because they're for different data sets):

enter image description here

Now, the thing in green is called the likelihood ratio test statistic. For sufficiently large sample size it has approximately a chi-square distribution.

The thing in red is called the p-value. It is not correctly defined in the Stata information you linked. It is the probability of getting a chi-square value at least as large as the one you observed if the null hypothesis were true. It is correctly defined in the first sentence here.

You decide significance by comparing the p-value with your significance level. (You haven't said what significance level you're using.)

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Since most of my parameter estimates were statistically significant for p=0,05, I thought my significance level is 0,05. But I do not really what the significance level of the likelihood ratio test is for me. Where could I find that out? The study I'm replicating uses Prob > χ2= 0.000 to show that their model is statistically significant. Thus I think my model should be significant as well. Is this conclusion reasonable? $\endgroup$ – user37838 Jan 25 '14 at 15:35
  • $\begingroup$ You don't discover what your significance level is - it's like when you drive your car somewhere; you don't find out what your destination is after you get in and drive about a while, it's something you figure out before you pick up your keys. The significance level - your type I error rate - is a deliberate choice you make before you begin your analysis (and hopefully, before you even begin to collect data). $\endgroup$ – Glen_b Jan 25 '14 at 22:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.