I am fitting a basic generalized linear model (binomial) to some survey data that has a small sample size (N=376). The model my colleague has fit has 23 model parameters. Two or three have p-values less than 0.1.
There are theoretical reasons to expect that those particular independent variables are linked to the dependent variable, so that's good. But, with so many independent variables included in a model, particularly with a reasonably small sample size, is it not the case that the risk of a type I error increases? In other words, putting so many model parameters into the model, are we not increasing the chances that some parameters will present themselves as statistically significant, just out of random chance?
To test this, would it be generally adviseable to begin removing non-statistically significant independent variables from the model to see if the remaining parameters remain significant?
I think I'm sensitive to the danger of chasing the stars of statistical significance, but I want to be sure my logic and modelling strategy is correct.