0
$\begingroup$

I am trying to sort out the unique effect of various environmental predictors on species occurrence (presence/absence data). I have been running glm models in R with family=binomial. Most of my variables have very highly significant P values, but I have a large data set (~8000 data points), so this is maybe not so surprising. I have also been calculating the percentage deviance explained using the BiodiversityR package. Some of my very significant variables based on the P value have very low explained deviance (1% or so).

Is there a rule of thumb for saying a predictor is not significant based on the deviance explained? Any other tests I should be including? I wanted to look at each variable on its own, and then test out specific interactions, but only if the variable was "important" in a model where it was the only predictor.

$\endgroup$
4
$\begingroup$

You are confusing two things:

Statistical significance and practical importance (effect size). As you note, significance is partly an effect of sample size.

However, % of deviance explained is an effect size measure. Such a measure is not either significant or not sig; it may be important or not important. Unfortunately, any guidelines as to how big a % is how important is going to be context dependent.

e.g.

If you reduce the rate of airplane crashes by 1 in 100,000 flights, that would be huge. If you reduce the rate of acne by 1 in 100,000 faces, that would be preposterously trivial.

Similarly, in some areas of physics, R of .9 is disappointingly small while in some areas of social science, R of .9 is so large as to lead one to think something is wrong.

The same will occur with any effect size measure, including % of deviance explained.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the confirmation! I am trying to compare the importance, and the possible interaction, of environmental variables to explain species distribution, using several species as the response variables (you could think of these as my reps). The species all have different numbers of presences, and different relationships with the predictive variables. Could you suggest the "best" metrics of effect size to do this? Area under the ROC curve, % deviance explained? None of these models are very complex I am only considering two variables at a time, with possibly cubic terms and interactions. $\endgroup$ – Frieda May 28 '14 at 19:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.