0
$\begingroup$

I have calculated the churn probability score for every customer id using glm model. So, I have a data frame with every customer id and its churn probability score. Ex cust_id 1 has a score of 0.11 which means there is 11% chance that cust_id 1 will churn. I have a older model that does the same thing and provides similar output. I want to check how similar are the 2 models predicting the churn scores. I used the two models coefficient to calculate the churn score on the same dataset. So now I have a dataframe the has customer id, churn score from new model and churn score from old model. I want to compare how similar are the scores from the old and new model. Is there any statistical test/method way to do this?

I calculated the difference between the score and tried to plot a histogram to check if it is centered at 0.

$\endgroup$
3
  • $\begingroup$ What are the differences between the two models? $\endgroup$ Sep 22 at 14:32
  • $\begingroup$ The source of gathering data is different. The older model used the on-premise data while the new model is using the cloud data. Predictor variable are the same in the 2 models. $\endgroup$ Sep 22 at 14:36
  • $\begingroup$ You could start by drawing a chart of the new score plotted against the old score for each customer, which may give you some basic information $\endgroup$
    – Henry
    Sep 22 at 14:42

1 Answer 1

0
$\begingroup$

Because the two models are ostensibly the same, just with different sourcing of the variables, I would just plot the predicted probabilities against one another and look for large discrepancies. You are likely interested in practical differences, not statistical differences.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.