I have some data (eg. Titanic) and I want to use logistic regression to predict probability of survive. I have problem to understand the difference between model development and model validation. Firstly we have to choose which variables use in our model and to do it we need to have tool which shows how good is our model. What is this tool in logistic regression? If I have for example 10 independent variables, should I delete some insignificant variables or it is okay to keep it in the model? How to choose which varables delete? We also have to choose in some way "threshold" to decide if someone survive or not. We do it during model development or during model validation? When model development ends and starts model validation?

  • $\begingroup$ If you are following the training-validation-testing paradigm then you can develop a variety of models (such as which variables to include and how) and then use (cross-)validation to choose which to include: the validation step sees how different ways of tuning your model might affect prediction accuracy on pseudo-out-of-sample data. So here, model development leads to the range of models to consider and model validation chooses which one of these to use, with model testing at the end as a one-off check on the predictive accuracy of your final model. $\endgroup$
    – Henry
    May 4 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.