0
$\begingroup$

I am trying to find the strength of signal over a background using a continuous variable, whose distributions are known for the expected signal, the expected background, and the observed data, along with corresponding uncertainties. Now if I understand correctly, due to a large number of observations, the distribution needs to be split into bins and the significance is to be evaluated using a binned maximum likelihood on the histograms.

Now the binning of the histogram, of course, can be done in many ways, the two extremes being just one bin for the whole data, and every datapoint getting its own bin. Furthermore, bin widths can be made variable, or equivalently, a different function of the variable can be discretized into equal-width bins (x^5 instead of x).

Is there a way to choose the binning in a way such that the net significance of the signal across all bins is maximized?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ If the reason for binning is computational speed, why not just take the max number of bins you can afford? $\endgroup$
    – kjetil b halvorsen ♦
    Commented Feb 23, 2022 at 0:48
  • $\begingroup$ Binning for any reason other than computational speed is a bad idea; use as many bins as you can. Each bin should have an equal number of observations to minimize the amount of information you lose by binning. $\endgroup$
    – Closed Limelike Curves
    Commented Feb 23, 2022 at 6:17
  • $\begingroup$ Binning is necessary mostly due to computation of bin-by-bin uncertainties taking all correlations into account. Besides, with the "expectation" distributions being statistically limited (I am using a MC simulation), the predictions at the tails of the distribution are likely to have large statistical uncertainties. At least at the edges, I would need to have reasonably large-width bins to have a reliable signal and background prediction. $\endgroup$
    – dan
    Commented Feb 23, 2022 at 22:56

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.