A basic question about a randomized test involving the error type I

I have a basic question in the context of testing statistical hypotheses, more specifically, about randomized tests. Suposse that I have two actions (altenatives) about a certain unknown parameter $$\theta \in \Theta$$: the Null ($$H_0$$) and altenative hypotheses ($$H_1$$).

In this case, the sample space is $$(0,15) \subset \mathbb{R}$$. We know that the critical function is given by $$\phi(x) = P(reject \, \, H_0 \,\,|\,\, x \, \, observed)$$

I don't know exactly if this definition really involves conditional probability. Suposse I have the following critical function

$$\phi(x)= \begin{cases} 0, \quad x \in (0,2)\\ p, \quad x \in (2,10)\\ 1, \quad x \in (10,15)\\ \end{cases}$$

I can not understand why

$$P( reject \,\, H_0 \,\,|\,\, H_0 \,\, is\,\, true)\, = 0 \times P(x \in (0,2)) + p \times P(x \in (2,10)) + 1 \times P(x \in (10,15))$$

The right side looks a lot like a some expectation. But I can not understand the equality.

This "critical function" $$\phi$$ is a tool to make a decision. Given the value $$X$$ of a test statistic, independently observe a uniformly distributed variable $$U$$ (supported on $$[0,1]$$). Reject the null hypothesis when $$U \le \phi(X);$$ otherwise, do not reject it.

To be explicit, given an experimental outcome $$\omega$$ (which is typically a random sample from the population), one often writes $$X(\omega)$$ for its test statistic. We may, if we choose, consider the "outcome" to include the random value $$U$$, allowing us to write $$U(\omega)$$ for the random value used in the decision process.

In these terms, the event "reject $$H_0$$" consists of all outcomes $$\omega$$ for which $$U(\omega)\le \phi(X(\omega)).$$ You can visualize this condition as a set in the $$(X,U)$$ plane. The set is (of course) limited to the supports of $$X$$ and $$U:$$ that is, it's a subset of the rectangle $$[0,15]\times [0,1].$$

With this plot in hand, making a decision is clear and simple: compute $$X$$ from the data and generate $$U$$; plot the point $$(X,U)$$ on the preceding figure; and reject $$H_0$$ if and only if the point lies within the shaded region.

1. The probability $$\Pr(\text{reject }H_0\mid X)$$ really is a conditional probability. That is explicit now that "reject $$H_0$$" has been exhibited as an event and $$X$$ has been described as a random variable.

2. In this problem the shaded region naturally decomposes into three disjoint rectangles according to how $$\phi$$ has been expressed: $$\text{Reject } H_0 = [0,2) \times {0} \cup [2,10)\times [0,p] \cup[10,15] \times [0,1].$$ (This decomposition is not unique: a look at the figure may suggest other ways to carve the region up into rectangles.) The chance of any such rectangle is easy to compute using the axioms of probability:

• Because the rectangles are disjoint, their chances add.

• Because $$X$$ and $$U$$ are (by construction) independent and $$U$$ has a uniform distribution, the chance of any rectangle $$[a,b)\times [c,d]$$ is the product of the chances, $$\Pr\left([a,b)\times [c,d]\right) = \Pr(X\in [a,b))\,\Pr(U\in [c,d]) = (d-c) \Pr(X\in [a,b)).$$

Applying these observations yields \eqalign{\Pr(\text{Reject }H_0) &= \Pr(X\in [0,2))\times 0 + \Pr(X\in [2,10))\times (p-0) + \Pr(X\in [10,15])\times (1-0) \\ &= 0\Pr(X\in [0,2)) + p \Pr(X\in [2,10)) + \Pr(X\in [10,15]).}

3. The foregoing can be viewed as an expectation in exactly the same way any probability is an expectation, by introducing the indicator function. Define $$I_{\text{Reject}}(\omega) = \left\{ \matrix{1 & H_0\text{ is rejected}\\0 & \text{otherwise.}}\right.$$ Then by definition $$E[I_{\text{Reject}}] = 1\,\Pr(H_0\text{ is rejected}) + 0\,\Pr(H_0\text{ is not rejected}) = \Pr(H_0\text{ is rejected}).$$

• Dear, I have some questions yet. First, I do not know what is the relationship between U and p. Is p the realization of U? Second, how about $P( reject \,\, H_0 \,\,|\,\, H_0 \,\, is\,\, true)\,$? I think you only show how to get $P( reject \,\, H_0 \,\,)$ – Fam Aug 14 '19 at 16:50
• (1) I believe I have been very clear about the relationship, but let me try once more: $U$ is a random variable. $p$ is a number. You compare one to the other to make your decision. (2) Your question title explicitly asks about "Type I" errors. That means the only thing you are asking about is the chance of rejecting the null when it is true. – whuber Aug 14 '19 at 18:56