I had a dataset with binary data. I created a logistic regression model with continuous data on the x-axis and binary data that has values 0 and 1 on the y-axis. Then I plotted my model, a line that shows the possibility of belonging to the '1' value on the y-axis. The probability line seems fine, but how can I test the accuracy of this model?

  • $\begingroup$ The tool of choice is a proper scoring rule. More information in the tag wiki. $\endgroup$ Jan 28 '21 at 10:30
  • $\begingroup$ @StephanKolassa yes it does, thank you. But I have another question. What is the perfect model for estimating probability of two events? $\endgroup$
    – atilla
    Jan 28 '21 at 10:33
  • $\begingroup$ I don't think there is a "perfect model". If there were one, then hordes of statisticians and data scientists, whose job is to find a better model than the current one, would be out of a job. Believe me, the people who pay my salary would love to know there is one perfect model, and they could stop spending so much money on my! Also see George Box's quote: "All models are wrong, but some are useful." $\endgroup$ Jan 28 '21 at 10:35
  • $\begingroup$ In that case, can I use logistic regression for probability predicting? $\endgroup$
    – atilla
    Jan 28 '21 at 10:36
  • $\begingroup$ Sure! Or a CART, or a Random Forest, or any flavor of NN. But starting with a very simple logistic regression should definitely be the first step! (The even earlier zeroth step should be to just take the average incidence of your target class in the training sample, which is equivalent to a logistic regression without any predictors. It may well be that your models can't even improve on that. Assess everything on a holdout sample.) $\endgroup$ Jan 28 '21 at 10:38