1
$\begingroup$

Is it legitimate to calculate confidence intervals for bounded measures like classification accuracy, AUC or correlation based on variance? Is it all right to say that the confidence interval is, for example, 0.99±0.2, although the upper limit is out of range of the measure? Shouldn't the confidence interval be asymmetrical for the bounded measures if the estimated value is close to one of the bounds?

All the literature I could find (like Computationally efficient confidence intervals for cross-validated area under the ROC curve estimate, Accuracy Assessment) assumes that the confidence intervals are symmetrical for the whole range, possibly because of the central limit theorem. But I am having difficulties to accept this assumption, because, for example, the upper 95% confidence interval for a vector that contains thousand times 0.99 and once 0.1 is 1.04. And such confidence interval is unnecessarily wide on the right side. Do you have some reference to literature: where is this topic discussed?

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Your misgivings are justified. It's between awkward and illogical to project confidence intervals into impossible territory. There is a great deal of literature on this. For example, it's been accepted for a century that confidence intervals for correlations be based on an atanh scale. See en.wikipedia.org/wiki/Fisher_transformation (an article which follows a multitude in using inappropriate arctanh notation: see e.g. stata-journal.com/sjpdf.html?articlenum=pr0041 on that point). $\endgroup$
    – Nick Cox
    Commented Dec 24, 2015 at 16:14
  • 3
    $\begingroup$ What's more, there's much literature on intervals for binomial proportions that also respects this issue, e.g. projecteuclid.org/euclid.ss/1009213286 $\endgroup$
    – Nick Cox
    Commented Dec 24, 2015 at 16:15

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.