3
$\begingroup$

I have often heard it claimed that there is a duality between hypothesis tests and confidence regions:

If $\langle\Omega,\mathcal{F}\rangle$ is a measurable space and $\Theta$ is a parameter space for a family of probability measures $P$ on $\mathcal{F}$, then a confidence region procedure is a function $C$ from $\Omega$ to the power set of $\Theta$ (taking an observed $\omega\in\Omega$ to a set of parameter values "consistent" with $\omega$), and a hypothesis test procedure is a function $T:\Theta\to\mathcal{F}$ where $T(\theta_{0})$ is the acceptance region for the null hypothesis $\theta=\theta_{0}$.

The duality is generally supposed to work like this: If $C$ is a confidence region procedure, then $C^{\ast}:\Theta\to\mathcal{F}$ defined by $$C^{\ast}(\theta_{0})=\{\omega\in\Omega \,\vert \theta_{0}\in C(\omega)\}$$ is the test procedure associated with $C$, and similarly for $T^{\ast}$: $$T^{\ast}(\omega)=\{\theta_{0}\in\Theta\, \vert \omega\in T(\theta_{0})\}.$$

It is immediately clear that, for instance, $(C^{\ast})^{\ast}=C$, and similarly for $T$.

So far, so simple, except that I am having trouble with this definition of $C^{\ast}$. In the general case, there is no reason to assume that $C^{\ast}(\theta_{0})$ is actually an element of $\mathcal{F}$, which means that it may not be meaningful to assign a probability to rejection or acceptance of the null hypothesis.

There are certain special cases where everything works fine. If $\Omega$ is countable, for example, it is reasonable to think that $\mathcal{F}$ is the entire power set of $\Omega$. Likewise, if $\Omega$ and $\Theta$ are intervals and we impose some convexity and continuity conditions, everything turns into an interval, and things work out just fine.

So, what I'm asking is whether there is any agreed upon set of criteria beyond the definitions I wrote above to make this duality work properly? Or is the duality only ever understood to apply in well behaved one dimensional contexts?

$\endgroup$
1
$\begingroup$

I've always thought that as long as there exists a valid/correct hypothesis test for a set of parameters, then you have the necessary basis for forming a confidence region. This presentaiton has a nice discussion on turning hypothesis tests into confidence regions. Its not as simple as 1-parameter CIs, but you can turn any hypothesis test intoi a CR. Note that the resulting regions need not be "nice" at all, as long as they include all parameter values that would not be rejected given the data and significance level for the underlying hypothesis test.

$\endgroup$
  • $\begingroup$ That seems right; my problem is going the other direction, from a CR to a test. And you can't have a one directional duality! Generally there doesn't seem to be any required structure on the parameter space, so it doesn't matter if the CR isn't nice. But all sorts of bad things happen if you try to construct a hypothesis test with a non-measurable acceptance region. $\endgroup$ – Unwisdom Jan 26 '14 at 5:39
  • $\begingroup$ @Unwisdom well, given a CR you can always do the hypothesis test: $\theta \in CR$, then take the data, form a CR and check if your parameter is in the region or not and reject or not based on that. BTW: can you give a practical example of a non-measurable acceptace region generated by a test? $\endgroup$ – user31668 Jan 26 '14 at 18:42
  • $\begingroup$ You're right that for a specific $\theta_0$ and given a confidence region procedure $C$ and an observed $\omega\in\Omega$, it is always possible to test the hypothesis $\theta=\theta_0$. But if the acceptance region for this set is not measurable, then it is not possible to calculate the power of the test. In response to your second point, the short answer is no. This doesn't seem to be a problem that arises naturally all that often (since non-measurable sets seldom arise naturally). It's easy to construct an example though - just make $\mathcal{F}$ coarse. $\endgroup$ – Unwisdom Jan 27 '14 at 1:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.