1
$\begingroup$

I have a result of a measurement, A(1.029) ± s(0.048), where s is one standard deviation. In a theory, A is restricted to be in range of [0,1]. The question is what is the correct way to set a lower limit on A at 95% Confidence level?

A similar discussion: Can one-sided confidence intervals have 95% coverage

Thanks, MJK

Improve this question
$\endgroup$
8
  • 5
    $\begingroup$ If your measurement is 1.029 but A has to be between 0 and 1, I'd say you have problems aside from any confidence interval. $\endgroup$
    – Peter Flom
    Commented Sep 26, 2019 at 14:55
  • 1
    $\begingroup$ It looks like you need a new theory since A is measured to exceed [0,1]... $\endgroup$
    – Ron Jensen
    Commented Sep 26, 2019 at 14:56
  • 2
    $\begingroup$ @Peter (and Ron and EdM) It's not in the least unusual for measurements to lie beyond theoretically prescribed bounds. For instance, many low-level radiological measurements are negative. Thus, there's nothing mysterious or wrong about either an estimate or a confidence interval including unrealistic numbers. $\endgroup$
    – whuber ♦
    Commented Sep 26, 2019 at 17:11
  • 1
    $\begingroup$ What do you know about the sampling distribution of A? For example, if you measured A over and over again, would the values of A resemble a Normal distribution with SD = 0.048? This is equivalent to saying you have Normally distributed measurement errors. If you are willing to assume your measurement error is ~ N(mean=0, sd=0.048), then (1.029-1.644854*0.048, INF) is a 95% one-sided confidence interval. I'm hesitant to suggest that as an answer without knowing more about the problem and how realistic that assumption is. $\endgroup$
    – Robert Alan Greevy Jr PhD
    Commented Sep 26, 2019 at 17:35
  • 1
    $\begingroup$ @Dave You seem to be missing the idea that $A$ is a measurement with some potential error. For instance, $A$ could be an electronic signal sent by a photographic sensor. Noise in the electronic system could (and does) cause $A$ to exceed the maximum value the sensor theoretically could register, which we can take to equal $1.$ That noise might be well understood (and thus its variance, and possibly its full distribution, may be known to high accuracy). What limits should one put on the true signal when a value of $1.029$ is observed? $\endgroup$
    – whuber ♦
    Commented Sep 27, 2019 at 14:18

1 Answer 1

Reset to default
0
$\begingroup$

I think what you can do is choose the quantile such that upper limit is $1$ and then back calculate the lower limit.

Let us first state your problem in a more formal way:

Let you have $n$ independent measurements $a_i$ for $A$. Assume the measurement is done with an error (which has expected value of zero) and therefore $a_i$ is random with some distribution around $A$ (alternatively normal distribution can be assumed but for large $n$ that is unimportant).

Now let $\bar{a}$ and $s$ be the mean and standard deviation of the sample $\{a_i\}_{i=1}^n$.

Using CLT and Slutsky's theorem, we have, for large $n$: (see this for details)

$\dfrac{\sqrt{n}(\bar{a}-A)}s \xrightarrow{d} Z \sim N(0,1)$

From this you get, confidence interval on $A$. Now say you objective is to get the most meaningful confidence interval on $A$. From the distribution of the statistic we have created we have:

Probability that the random interval $(\bar{a}-1.96s/\sqrt{n},\bar{a}+1.96s/\sqrt{n})$ contains $A$ be denoted by: $Pr((\bar{a}-1.96s/\sqrt{n},\bar{a}+1.96s/\sqrt{n}) \ni A)$

Therefore, we have from our result that for large $n$:

$Pr((\bar{a}-1.96s/\sqrt{n},\bar{a}+1.96s/\sqrt{n}) \ni A)=0.95$

The number $1.96$ came from $F^{-1}(0.975)$, where $F(.)$ is the cdf of normal distribution.

Now let $q_2$ s.t. $\bar{a}+q_2s/\sqrt{n}=1$.

Assume further that $F(q_2)>0.95$ ...................(A1)

Further, calculate $q_1$ s.t. $F(q_2) - F(q_1)=0.95$

This means, that $Pr(Z \in (q_1,q_2]) = 0.95$

Therefore, $Pr((\bar{a}-q_1s/\sqrt{n},\bar{a}+q_2s/\sqrt{n}] \ni A)= 0.95$

$\implies Pr((\bar{a}-q_1s/\sqrt{n},1] \ni A)= 0.95$

So $(\bar{a}-q_1s/\sqrt{n})$ becomes a more meaningful lower limit for $A$.

EDIT (I am retaining the above answer for sake of completeness and clarity)

I realized that your $\bar{a}>1$. In such a case, $q_2$ will come out to be negative to satisfy $\bar{a}+q_2s/\sqrt{n}=1$. And because of this $F(q_2)<0.95$. In fact, $F(q_2)<0.5$. So A1 is not satisfied. Instead what you can do is the following:

Let $q_1=1.64$. This means that $Pr(Z\leq q_1)=0.95$.

This means, that $Pr((\bar{a}-q_1s/\sqrt{n},\infty) \ni A)= 0.95$

$\implies Pr((\bar{a}-q_1s/\sqrt{n},1] \ni A) + Pr((1,\infty) \ni A)= 0.95$

But we know from theory, $ Pr((1,\infty) \ni A)=0$.

Therefore, $Pr((\bar{a}-q_1s/\sqrt{n},1] \ni A)=0.95$

Improve this answer
$\endgroup$
6
  • $\begingroup$ When $\bar a \gt 1,$ $q_2$ is negative and your procedure falls apart. A more basic reason this doesn't work can be ascertained by contemplating slightly different data. Suppose the value $1.2$ is observed for $A,$ which still is believed to have a standard deviation of $0.048.$ Your lower confidence limit will be substantially greater than $1:$ in what sense is this "more meaningful" (as you claim) or even meaningful at all? $\endgroup$
    – whuber ♦
    Commented Sep 27, 2019 at 13:14
  • $\begingroup$ See the part after EDIT in the answer. $\endgroup$
    – Dayne
    Commented Sep 27, 2019 at 13:22
  • $\begingroup$ Thank you--I hadn't noticed that you were trying to address this issue. However, I can't find any resolution there: exactly what are you claiming about the CI when $\bar a \gt 1$? Please note that when $q_1=-1.64,$ $\bar a - q_1 s/\sqrt{n}$ is even larger than $\bar a,$ making it impossible for your probability assertion to be correct. $\endgroup$
    – whuber ♦
    Commented Sep 27, 2019 at 13:28
  • $\begingroup$ Right. That's a mistake. Let me correct that. Thanks for pointing out. $\endgroup$
    – Dayne
    Commented Sep 27, 2019 at 13:30
  • $\begingroup$ Small edits. Hope it's correct now. $\endgroup$
    – Dayne
    Commented Sep 27, 2019 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.