0
$\begingroup$

I am trying to convert a continuous measurement of a patient’s bone mineral density to a risk score which I will display to the user with the corresponding observed prevalence (observed probability of having low BMD) in a test population. I am wondering if there is a scientific way of determining the optimal number of bins (N) or if binning is even required. Do I need to perform ordinal regression?

Am I required to perform some kind of calibration on the validation set to help determine this or is it as simple as “I want 10 bin”? If calibration is required, what metric helps determine optimal number of bins?

Should I make sure each bin has the same number of samples or should it be equal ranges of the continuous score?

$\endgroup$

1 Answer 1

3
$\begingroup$

Binning throws away data. It is often good to look at a histogram, which is binned, to understand your data.

If you have a well-founded hypothesis about the underlying distribution, then it is OK to use a parametric approach, and estimate the Probability Distribution Function, and you can do all that with binning.

But if you don't, then non-parametric methods are preferable, e.g. the Kolmogorov-Smirnov test. In this case you use the Cumulative Distribution Function. A graph of the CDF is often harder to read, but statistically better practice. In particular, there is no unbiased estimator of the PDF.

In your case, a graph of the CDF has as x axis bone mineral density and as y axis % of the population with an actual average density lower than x - this would seem to be a quite natural graph to plot in your context.

$\endgroup$
4
  • $\begingroup$ Thanks for the answer. Does the non-parametric method assume the test population is the same distribution as the future population it will be deployed in. This is not quite the situation I have (our test set is quite heavily enriched). The target population is the normal population and we have currently selected our bins to be in 0.5 T-Score increments. I understand that binning can throw away some data but if the bin widths are selected to be less than the 95% confidence intervals of the point estimates would you agree this is less likely to have a negative impact? $\endgroup$
    – Cicce19
    Apr 6, 2022 at 18:45
  • $\begingroup$ "if the bin widths are selected to be less than the 95% confidence intervals of the point estimates" then you have ensured that each point is likely to be in the right bin - if it is actually central to the bin. But for a point near the bin boundary, you can never be sure. $\endgroup$ Apr 8, 2022 at 7:35
  • $\begingroup$ More importantly, by reporting only which bin a point is in, rather than the measured value, you are throwing away data. $\endgroup$ Apr 8, 2022 at 7:36
  • $\begingroup$ Every statistical method assumes that the test population is representative of deployment population. With one exception: many methods allow you to use a "sample weight" vector, to adjust this. I assume that your future population is "people who are suspected to have osteoporosis". A training set of students will clearly not help you build the right model - you need to address this whether your methods are parametric or nonparametric. $\endgroup$ Apr 8, 2022 at 7:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.