I was reading a very insightful paper from booking.com published at KDD 2019 150 successful Machine Learning models: 6 lessons learned at Booking.com. The paper shares practical lessons from deploying ML models.
One interesting part is how they use the distribution of the deployed model's predictions to diagnose the model. Specifically, the followings are several different examples of the distributions over the deployed model's scores (which is from 0 to 1).
The paper says
- the first case is possibly due to the high bias or high Bayes (error) in the model
- the second case is possibly due to wrong scaling in a feature or false outliers
- the third case (non-smooth curve) is possibly due to the deploying being too sparse
- the fourth case is an ideal case where there is bimodal at 0 and 1
The paper doesn't provide the details of the diagnosis. Can somebody elaborate on these?