1
$\begingroup$

I am a beginner in statistics and R.

I want to run a binary logistic regression to understanding (modeling) factors affecting nest-site selection in a bird species. I have Presence/Absence(0,1) data and 13 predictors.

I wanted to use the step-wise method before (using stepAIC function in R), but luckily I found out on this site that it is an invalid method. So I fit the model with all the variables. But now I am confused in interpreting the results because the p-values of all of the variables are more than 0.10.

Unfortunately, I also cannot run a lasso method, because I have some categorical variables and also I am using splines for some continuous variables. I tried to use the lasso method with the "glmnet" package in R, but I was unsuccessful.

What method do you suggest for reporting significant variables?

$\endgroup$
8
  • 3
    $\begingroup$ Pick variables according to theory. $\endgroup$ Mar 4 at 23:47
  • 2
    $\begingroup$ Additionally, I'm not sure why people continue to believe that variables can reliably be selected from the data without an effect on the quality of the inference. Please, let go of the idea that this can be done (easily) and instead leverage on your knowledge of the science in which you participate. $\endgroup$ Mar 4 at 23:51
  • $\begingroup$ You mean, for example, I can consider a significance level of 0.2? $\endgroup$ Mar 4 at 23:57
  • 3
    $\begingroup$ Think of it this way. Suppose that your field has studied variable $x$ in the past. Many papers have been written on the importance of $x$ and there is a lot of theory -- not just modelling -- for why $x$ is important. Given this, would you include $x$ in your model? $\endgroup$ Mar 5 at 0:01
  • 1
    $\begingroup$ We’ve already determined step wise is to be avoided. I’d your predictors are new, there are other methods of determining which variables to include which do not depend on data. $\endgroup$ Mar 5 at 0:19

1 Answer 1

3
$\begingroup$

Just as a side note, because you are new to both R and logistic regression, I highly recommend reading through Practical Guide to Logistic Regression in R by Joseph Hilbe, which covers how to do logistic regression in R. Given you are new to statistics in general, Learning Statistics with R by Daniel Navarro should also precede this so you have a base understanding of how statistics work. I also strongly advise against using really complicated procedures like splines or lasso regression if you are new to stats, as it requires a solid understanding of a lot of statistical principals before employing them.

As Demetri already pointed out, the need to select predictors should be heavily theory-driven or at least based on some research. I find it difficult to believe there isn't some research on your idea or what predictors you need, even if it isn't directly comparable. For example, my field has decades of research on the effects of morphological awareness on reading ability. This has been teased apart a bazillion ways. Morphological awareness can be sub-typed by three varieties in European scripts, can be categorized as a single construct in written scripts like Chinese, and is sometimes lumped together with other predictors of reading using factor analysis.

Suppose I believe there is some element of morphological awareness that simply hasn't been tested yet. We can call it Factor M. Factor M hasn't been tested yet, but we as researchers believe it has some effect on reading. To study this, we include it into a regression using a novel measure to identify it's effects. We create a candidate model to the effect of something like this:

$$ \text{Reading Comprehension} = \beta_0 + \beta_1\text{Factor M} + \epsilon $$

Perhaps we also believe that while the effect of Factor M alone is meaningful on reading comprehension, perhaps the influence of a second variable is important to control for, so we also create a candidate model with this effect, defined below with Factor N as the control variable.

$$ \text{Reading Comprehension} = \beta_0 + \beta_1\text{Factor M}+ \beta_2\text{Factor N}+ \epsilon $$

With these two candidate models, we can test their effects. Thereafter we can perform model comparisons using AIC, BIC, etc. However, at no point have we invented variables out of thin air or just grabbed whatever variables we could. This is because of the following reasons:

  • It is a huge waste of time. In the case where we have 50 variables, we could spend forever trying to clean the data, check assumptions like linearity, estimate outliers, etc. Why waste all that effort when one can simply select predictors that have already been researched and are far more likely to yield results you believe exist? Better to save time on all of this by selecting only what you need.
  • It is very unscientific. This verges on HARKing (Hypothesizing After Results are Known), in that you are just applying whatever regression gives you some nice p values and then shipping your baked p-values to the nearest journal. Yet without hypothesizing before the fact what you achieved, you are conveying that you already knew this relationship existed when in fact you didn't. More importantly, you may capitalize on totally chance findings that will never be replicated with future testing.
  • The meaningfulness of the model may be questionable at best. Let's say we find that wind speed, the amount of oxygen I breathe, and the number of letters I type in a day all are significant predictors of math ability. Why? What could we possibly derive from such a model without at least some assumptions of what relationship actually exist? A regression of this variety would be pointless to entertain.

So to summarize and answer your main question:

What method do you suggest for reporting significant variables?

The answer is the normal way.

$\endgroup$
4
  • 2
    $\begingroup$ Some good wisdom. $+1.$ $\endgroup$ Mar 5 at 13:16
  • $\begingroup$ @Shawn What do you mean by "the normal way"? I assume you mean "normal" to be a synonym for "usual" and not for "Gaussian".. $\endgroup$ Mar 5 at 15:41
  • $\begingroup$ The usual way, not Gaussian. $\endgroup$ Mar 5 at 16:00
  • $\begingroup$ Thank you so much Shawn. It is better to use fewer variables in the model. $\endgroup$ Mar 6 at 10:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.