I am trying to compare results from a recent nuclear physics experiment with an existing result that was published 13 years ago. Both experiments measure a quantity X, which is effectively the rate at which some reaction Y happens when you "throw" two specific nuclei at each other. The experiments are entirely distinct (different labs, different measurement equipment, different backgrounds, different experimenters, different numbers of nuclei thrown at each other, etc).

The result from the recent experiment is X = 17 +/- 11, where the +/- 11 refers to a 90% confidence interval. More specifically, the PDF for X is a normal distribution centered at 17, and the range (6,28) spans 90% of the total area of the distribution.

The previously published experiment reports an upper limit of X < 15.2, again at the 90% confidence level. Fortunately, the authors were thorough enough to state specifically what this means: the PDF for X is a normal distribution with a mean of -1.0 and a standard deviation of 9.619. The authors arrived at the upper limit X < 15.2 because (0,15.2) contains 90% of the total area of the X >= 0 portion of the PDF (note that X < 0 would be an impossible result since we are dealing with the rate of a physical process, which cannot be a negative number).

To clarify further, the reason the authors of the second study would have arrived at a normal distribution with mean -1 (even though negative values are physically impossible) has to do with backgrounds. The goal of the experiment is to count the number of times reaction Y happens. But due to imperfect measurement techniques, whatever number you count includes both the reaction you are looking for (the "signal"), as well as a background. Both the signal and background are Poisson distributed, but commonly approximated as normal (with standard deviation equal to the sqrt of the mean). The expected background rate can be estimated from other experiments, and subtracted from the total rate to give the estimated signal rate.

In the experiment in question, the estimated background rate was larger than the total observed rate, giving a mean signal rate less than zero. Nowadays there are established ways to deal with this sort of thing (e.g. Feldman & Cousins, Phys. Rev. D 57, 3873). But at the time, the authors had to come up with their own technique, so they decided to use the technique I explained above, setting the upper limit at the value that spans 90% of the area of the positive portion of the normal distribution.

My question is as follows: I would like to make some sort of statement about the "consistency" between the two experimental results. At a minimum I would like to make a qualitative statement, e.g. "the results from the two experiments are (or are not) consistent". But I am also interested in understanding what, if any, quantitative statements it would be possible to make to compare the two results.

  • 1
    $\begingroup$ Since negative values are not possible why do you say that the distribution is normal? How do you even know that it is symmetric approximately? $\endgroup$ – Michael Chernick Aug 2 '17 at 22:24
  • $\begingroup$ I added some more text to the post to hopefully answer @MichaelChernick's question. $\endgroup$ – GAC Aug 3 '17 at 4:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.