Logistic Regression Model building (dropping p-values)

Question

I'm researching factors that influence how bookable a certain room is with data similar to that of Airbnb. The idea is to look at what has been filled out in a listing of a room (i.e. utilities included, kitchen shared/private, toilet shared/private...) to make a prediction on the probability of booking for a room, as opposed to just taking the average booking. A secondary (and more applied goal) would be to say that certain criteria can be made mandatory as they make the room x-times more bookable. So by including x,y,z the room would be i.e. double as likely to get booked. I have experience with regression but am fairly new to logistic regression, so I'm not 100% sure if this is the most sensible use of this method to get to the above mentioned outcome.

To establish the probability of a booking I have ca. 40 possible criteria that could be predictors in the model which are taken from the listing process of roughly 7000 rooms (I can 'zoom' into different cities and compare to adjust for confounding factors), and after analyzing them and basing it on previous knowledge I came up with around 7-8 that should have the largest effect size.

I figured I would use logistic regression to build the model as I have both continuous predictors and categorical ones, and of course my categorical dependent (booking: either 0 no booking, or 1 booking). My problem is that if I test the significance of those individual predictors on the dependent (booking) they are all significant, but once I start fitting them into a model together the significance drops and only 2 or 3 (and not even the ones with the largest effect size) remain. I assume this is often due to multicollinearity (i.e. bed & desk in a room are usually both filled out together if they happen to be filled out). Another issue is that the data varies a bit across datasets, so some predictors are significant in one dataset (e.g. London), and then they are insignificant when looking at the world dataset. I suppose as this is not an experimental setting it makes it all a bit more complicating.

My questions are:

1. Is logistic regression the most appropriate method?
2. How much do p-values matter when assessing whether a predictor should be fit into the model (I read on some other posts that it shouldn't be your all-or-nothing criterion in selecting predictors, but rather that you should look at changes in pseudo R^2 and goodness of fit). In the end even if the lower bound of the CI of the Odds is below 1 it shouldn't matter to much as in reality if I think about it I doubt adding useful information such as specifications of the kitchen will decrease the bookability of a room.

Btw I'm running it on SPSS

Thanks a lot in advance.

Answer 1

Question 1: Logistic regression should certainly work here: you have a binary outcome (booked or not) and you have 40 predictors. You could consider adding quadratic terms (squaring all your continuous variables and adding them as extra predictors) or interaction terms (multiplying your predictors with each other).

If you feel like testing other methodologies and your focus is on creating a good prediction model, you could look into, e.g., support vector machines, decision trees, random forests. I would suggest the caret package for R (http://topepo.github.io/caret/index.html) for this. However, this will be some extra work but performance might increase. Also, some of these models might be harder to interpret. If you are really just looking into finding the predictors, logistic regression is already a good choice.

Question 2: p-values give good indications for individual variables but, as you said, as soon as you have correlating predictors in a model, they are harder to interpret.

If you are looking into building a good prediction model: I would evaluate models by their performance in a cross-validation (not sure whether SPSS can do that?). Take something like 70% of your dataset and test multiple combinations of variables in a cross-validation. You can evaluate your models using different metrics: Area under the ROC-cure (AUC) for discrimination, mean-squared error for minimizing the prediction error, etc. Choose a metric and take the model with the best metric, test it on the 30% of 'unused' data - this is the estimate of how your final model performs. On top of that, you can choose to use penalized logistic regression (logistic LASSO, elastic net) to select predictors for you (again, extra work to learn this but it might save you plenty of time in the future). In that case you would have to tune the penalization parameters in the cross-validation instead of trying different combinations of predictors.