3
$\begingroup$

I was thinking about CI and subjective Bayesian and I have following two questions:

  1. If a subjective (not objective) Bayesian would care if her predictions don't do well in the real world.

  2. A classical statistician would not care if her confidence statement is (obviously) wrong for a given data set (as in Welch's Paradox, where conditioning is on ancillary statistics leads to the resolution of pathological behavior).

I think my answer for 1. is YES and 2. is NO.

But I don't know if I am thinking along the right lines. Can I have some more insights?

UPDATE

Welch's example: This example works for any $n$, but we will take $n=2$ for simplicity. $X_1, X_2 \sim U(\theta - 1/2, \theta +1/2)$ (iid), $\theta \in R$. This implies $X_1 - \theta \sim U(-1/2, 1/2)$ (iid). $(X_1 + X_2) /2 - \theta$ (note that this is NOT a statistic) has a distribution independent of $\theta$. We can choose c > 0 s.t. $Prob_{\theta} [-c \le (X_1 + X_2) /2 - \theta \le c] = 1- \alpha (~0.99)$, implying $((X_1 + X_2) /2 - c, (X_1 + X_2) /2 + c)$ is the 99% CI of $\theta$. The interpretation of this CI is: if we sample repeatedly, we will get different $(X_1 + X_2) /2$ and (at least) 99% times it will contain true \theta. But for a particular set of $X_1, X_2$, we can't say if the CI contains $\theta$. Now, consider the following data: $X_1 =0$ and $X_2=1$, as $|X_1 - X_2|=1$, we know FOR SURE (Prob =1) that the interval $(X_1, X_2)$ contains theta (one possible criticism, $P{|X_1 - X_2|=1} = 0$, but we can handle it mathematically and I won't discuss it). (Better details are in Pratt, 1961; Lehman, Chap 10, 2nd Edition, Prob 27, 28; Kiefer, 1977; Berger and Wolpert, 1988)

Thanks,

S.

Improve this question
$\endgroup$
5
  • $\begingroup$ Could you provide a link to the Welch paradox? $\endgroup$
    – user28
    Commented Nov 5, 2010 at 14:05
  • $\begingroup$ What's paradoxical about the Welch example? There's no contradiction in knowing \theta for sure. The classical statistician's explanation in this case might go, "I generated an interval using a method having at least a 99% chance of covering the true parameter (a priori). In this case, we know (ex post facto) that it indeed covers the parameter. Be happy!" $\endgroup$
    – whuber ♦
    Commented Nov 5, 2010 at 16:56
  • $\begingroup$ @whuber The paradox is about the CI only answering questions about repetitions, and not answering any questions for an observed data set. In the given data set ($X_1=0 and X_2=1$), we know with 100% prob that the $\theta$ lies between them. I agree with your comment. I guess I am not doing a great job of explaining, please see this link for what I am trying to say: books.google.com/… $\endgroup$
    – suncoolsu
    Commented Nov 5, 2010 at 18:09
  • 1
    $\begingroup$ To add a "proof" to some degree to why welch's example produces a "paradox", but in more general terms here on page 205-206 (page's 31-32 of the pdf file). This is becuase the distribution used in welch's example is a "location parameter" distribution (i.e. it is invariant under translations). It also shows that confidence intervals should really be based on the distribution of the sufficient statistic (which may be the whole data set), rather than just a convenient statistic. $\endgroup$
    – probabilityislogic
    Commented Oct 9, 2011 at 3:14
  • $\begingroup$ very well said @probabilityislogic . Actually we talked about this in a recent class, and what you say is true. $\endgroup$
    – suncoolsu
    Commented Oct 10, 2011 at 0:20

2 Answers 2

Reset to default
3
$\begingroup$

For many reasons you're right about 1. I certainly wouldn't heed the advice of someone who did not care about whether it is any good!

Number 2, as you have expressed it, does not characterize good practice. If there are possible datasets where a CI (or any decision procedure, for that matter) is clearly wrong, then that procedure is inadmissible by definition (because you can replace it by one that is at least as good and sometimes better, no matter what). Yes, inadmissible procedures are sometimes used in practice, but it can be argued that such procedures actually are admissible when we include the cost of performing the procedure itself within the loss function. (In other words, a quick and dirty method that works ok can be considered superior to one that requires extensive time and effort to calculate and works only a little better.) But in this hypothetical case, the "obviously" clause indicates it takes no effort to recognize some wrong intervals and replace them with better ones. Therefore, although I do think your reasoning is good, we should conclude that a thoughtful concerned "classical" statistician would indeed care: the answer should be YES to both questions.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I agree. (2) is a reason for conditioning on an ancilliary statistic. $\endgroup$
    – onestop
    Commented Nov 5, 2010 at 16:50
  • $\begingroup$ @onestop and @whuber I also agree about conditioning on ancillary statistic. But please let me know the fallacy in my line of thinking: A classical statistician will think that a datum (from the sample space) shouldn't trouble her (even though it doesn't look that it's wrong) as the CI is for repetitions and not for that particular datum. As whuber points out, a practical statistician would choose better CIs, but the point is more philosophical in my case. $\endgroup$
    – suncoolsu
    Commented Nov 5, 2010 at 19:42
  • 2
    $\begingroup$ @suncoolsu It might not look like it, but I did give an answer in a philosophical spirit! The point is that there is no problem or paradox with someone noting that the confidence actually is 100% in this particular case when they used a procedure designed to assure at least 95% confidence in any possible situation. In fact, something like this happens all the time with discrete data: in selecting a 95% CI procedure it may happen that the actual confidence is, say, always 97% or greater. A careful statistician would recognize and report that fact. $\endgroup$
    – whuber ♦
    Commented Nov 5, 2010 at 19:55
4
$\begingroup$

For the second question, I believe the answer is "Yes". I will quote Andrew Gelman here, "..in general there is no coverage guarantee because frequency properties depend on nuisance parameters which can only be ignored in some special cases of pivotal test statistics".

You can take a look at the following paper for some really nice discussion of relative merits and demerits of these procedures.

http://www.stat.columbia.edu/~gelman/research/published/badbayesresponsemain.pdf

V.S.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.