16
$\begingroup$

I was reading a paper recently that incorporated randomness in its confidence and credible intervals, and I was wondering if this is standard (and, if so, why it is a reasonable thing to do). To set notation, assume that our data is $x \in X$ and we are interested in creating intervals for a parameter $\theta \in \Theta$. I am used to confidence/credibility intervals being constructed by building a function:

$f_{x} : \Theta \rightarrow \{0,1\}$

and letting our interval be $I = \{ \theta \in \Theta \, : \, f_{x}(\theta) = 1\}$.

This is random in the sense that it depends on the data, but conditional on the data it is just an interval. This paper instead defines

$g_{x} : \Theta \rightarrow [0,1]$

and also a collection of iid uniform random variables $\{U_{\theta} \}_{\theta \in \Theta}$ on $[0,1]$. It defines the associated interval to be $I = \{ \theta \in \Theta \, : \, f_{x}(\theta) \geq U_{\theta} \}$. Note that this depends a great deal on auxillary randomness, beyond whatever comes from the data.

I am very curious as to why one would do this. I think that `relaxing' the notion of an interval from functions like $f_{x}$ to functions like $g_{x}$ makes some sense; it is some sort of weighted confidence interval. I don't know of any references for it (and would appreciate any pointers), but it seems quite natural. However, I can't think of any reason to add auxillary randomness.

Any pointers to the literature/reasons to do this would be appreciated!

Improve this question
$\endgroup$
3
  • 5
    $\begingroup$ (+1) This is called a randomized procedure. They are a standard part of the statistical estimation and testing framework, so you can rely on just about any rigorous textbook to provide explanations. Additional motivation for their use can be found in the game theory literature. $\endgroup$
    – whuber ♦
    Commented Dec 28, 2014 at 17:01
  • $\begingroup$ Thanks for the response. I realized after reading this comment that e.g. bootstrapping fits into this framework, but in that situation the reason for randomization is clear (you don't have access to f, just g). In my case, the authors explicitly calculate $f_{x}$, and THEN look at $g_{x}$. Although I have many stats textbooks, I don't see this anywhere... do you have a suggested text? $\endgroup$
    – QQQ
    Commented Jan 1, 2015 at 3:33
  • 3
    $\begingroup$ Actually, bootstrapping is not a randomized procedure. It is a determinate procedure whose approximate calculation is performed by means of random sampling. $\endgroup$
    – whuber ♦
    Commented Jan 1, 2015 at 5:31

2 Answers 2

Reset to default
4
$\begingroup$

Randomized procedures is used sometimes in theory because it simplifies the theory. In typical statistical problems, it does not make sense in practice, while in game-theory settings it can make sense.

The only reason I can see to use it in practice, is if it somehow simplifies calculations.

Theoretically, one can argue it should not be used, from the sufficiency principle: statistical conclusions should only be based on sufficient summaries of the data, and randomization introduces dependence of an extraneous random $ U $ which is not part of a sufficient summary of the data.

UPDATE

To answer whuber's comments below, quoted here: "Why do randomized procedures "not make sense in practice"? As others have noted, experimenters are perfectly willing to use randomization in the construction of their experimental data, such as randomized assignment of treatment and control, so what is so different (and impractical or objectionable) about using randomization in the ensuing analysis of the data? "

Well, randomization of the experiment to get the data is done for a purpose, mainly to break causality chains. If and when that is effective is another discussion. What could be the purpose for using randomization as part of the analysis? The only reason I have ever seen is that it makes the mathematical theory more complete! That's OK as long as it goes. In game-theory contexts, when there is an actual adversary, randomization my help to confuse him. In real decision contexts (sell, or not sell?) a decision must be taken, and if there is not evidence in the data, maybe one could just throw a coin. But in a scientific context, where the question is what we can learn from the data, randomization seems out of place. I cannot see any real advantage from it! If you disagree, do you have an argument which could convince a biologist or a chemist? (And here I do not think about simulation as part of bootstrap or MCMC.)

Improve this answer
$\endgroup$
12
  • 1
    $\begingroup$ Why do randomized procedures "not make sense in practice"? As others have noted, experimenters are perfectly willing to use randomization in the construction of their experimental data, such as randomized assignment of treatment and control, so what is so different (and impractical or objectionable) about using randomization in the ensuing analysis of the data? $\endgroup$
    – whuber ♦
    Commented Jan 1, 2015 at 19:58
  • 1
    $\begingroup$ @kjetil I think you might not have completed your statement about the sufficiency principle, it seems to have been cut off mid-sentence ("statistical conclusions should..."). $\endgroup$
    – Silverfish
    Commented Jan 1, 2015 at 22:33
  • 1
    $\begingroup$ (+1) But I think it's begging the question to invoke the Sufficiency Principle, the argument for which is that once you know the observed value of the sufficient statistic, taking any other aspect of the data into account is equivalent to introducing an extraneous random $U$. So someone proposing to do just that wouldn't give a fig for the Sufficency Principle. Also, see Basu (1978), "Randomization in Statistical Experiments", FSU Statistics Report M466 for a couple of randomized procedures proposed in earnest. $\endgroup$
    – Scortchi ♦
    Commented Feb 18, 2015 at 9:56
  • 1
    $\begingroup$ @whuber: It is a clear, principled argument that randomization in obtaining the data may be advantageous. (It breaks causal chains). What is that principled argument for using randomization as part of the analysis? $\endgroup$
    – kjetil b halvorsen ♦
    Commented Feb 18, 2015 at 10:24
  • 1
    $\begingroup$ Kjetil: It enables you to achieve the intended risk function, rather than accepting a risk function (often in the form of nominal size and power) that is not what you wanted. Moreover, if a procedure is "theoretically" useful then there certainly can be no objection to its use in practice, other than impracticability (which is usually not the case with randomized procedures). Thus your question should be turned on its head: the burden is on you to demonstrate there is something wrong with using randomized procedures. How do you accomplish that without contradicting yourself? $\endgroup$
    – whuber ♦
    Commented Feb 18, 2015 at 16:14
3
$\begingroup$

The idea refers to testing, but in view of the duality of testing and confidence intervals, the same logic applies to CIs.

Basically, randomized tests ensure that a given size of a test can be obtained for discrete-valued experiments, too.

Suppose you want to test, at level $\alpha=0.05$, the fairness of a coin (insert any example of your choice here that can be modelled with a Binomial experiment) using the probability $p$ of heads. That is, you test $H_0:p=0.5$ against (say) $H_1:p<0.5$. Suppose you have tossed the coin $n=10$ times.

Obviously, few heads are evidence agaist $H_0$. For $k=2$ successes, we may compute the $p$-value of the test by pbinom(2,10,.5) in R, yielding 0.054. For $k=1$, we get 0.0107. Hence, there is no way to reject a true $H_0$ with probability 5% without randomization.

If we randomize over rejection and acceptance when observing $k=2$, we may still achieve this goal.

Improve this answer
$\endgroup$
2
  • $\begingroup$ This is a nice explanation of the use of randomization, but it would be nice if it explained why we might be interested in attaining arbitrary $\alpha$ in the first place. Why is it a desirable goal? $\endgroup$
    – Silverfish
    Commented Feb 18, 2015 at 10:24
  • $\begingroup$ Well, that I guess brings us back to the history of statistics, when R.A. Fisher somewhat arbitrarily decided to work with a significance level of 5% to decide whether some initial evidence warrants further study. As we know, 5% has since morphed into a sort of gold standard in many fields, despite lacking good decision-theoretic foundation. $\endgroup$
    – Christoph Hanck
    Commented Feb 18, 2015 at 15:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.