Different ways of determining rejection region of a two sided test

Question

In a two sided test, assume the test statistic has been chosen to be $T(X)$ and the distribution of $T(X)$ under null hypothesis is also known to be $F$. Let the significance level be $\alpha$.

I can come up with two different ways to determine the rejection region:

1. $\{|T(x) - \mu| > c\}$. $\mu$ is the mean of the distribution $F$ of $T(X)$under null, and $c$ is determined by solving $$\inf_{c \geq 0} c$$ subject to $$P_{T(X) \sim F} (|T(X) - \mu| > c) \leq \alpha.$$ So the rejection region is symmetric around $\mu$.

2. $\{T(x) > c_1\} \cup \{T(x) < c_2\}$. $c_1$ and $c_2$ are determined by solving $$\inf_{c_1 \in \mathbb R} c_1$$ subject to $$P_{T(X) \sim F} (T(X) > c_1) \leq \alpha/2$$ and $$\sup_{c_2 \in \mathbb R} c_2$$ subject to $$P_{T(X) \sim F} (T(X) < c_2) \leq \alpha/2.$$ So the rejection region evenly split $\alpha$ to both sides.

Questions:

1. Am I correct that those two methods will agree when the null distribution $F$of $T(X)$ is symmetric around its mean $\mu$, and may not agree when $F$ isn't symmetric around $\mu$?

2. I was wondering what advantage and disadvantages these two methods have? Which one is recommended and when?

3. Are both methods used in some textbooks? If yes, references?

4. What are some other methods for two-sided tests? For example, can we generalized the second method by splitting $\alpha$ arbitrarily unevenly to the two sides?

5. Consider the relation between rejection region in testing and confidence interval. Are the above discussions also apply to confidence intervals?

Thanks and regards!

Answer 1

Question 1. Right, if $F$ is symmetric (and continuous: weird things may happen if some points have non-zero weight and you cut your interval right on them) you have $P(T(X)-\mu) > c = P(T(X)-\mu < -c)$, so you can split the rejection region into two unbounded intervals, each of probability $\alpha / 2$.

Questions 2,4. For any set $C$, as long as $P(T(X) \in C) = \alpha$, you're fine. You could even choose a region such as $[\mu-d,\mu + d]$ ! It is however not natural, because one would like your rejection set to include weird values of the statistic (values that are far from $\mu$) rather than normal values. Why? Because the test statistic is expected to be a measure of the distance between the data and the null hypothesis. For instance, in the lady tasting tea, the test statistic is the number of correctly classified cups. One would expect that the higher it is, the more the ability of the lady is proven. A weird rejection set could be designed, that could include the lady correctly classifying all cups, but not the lady correctly classifying all cups but one. It would be perfectly acceptable from a formal point of vue.

That insisting on having centered confidence intervals is not based on mathematics strictly can be seen easily: if $[a,b]$ is a centered confidence interval for parameter $\theta$, $[f(a),f(b)]$ has no reason to be a centered confidence interval for transformed parameter $f(\theta)$. For instance, say your data is sound power, and that you want to relate that to medical damages or sound perception; do you want intervals centered on the original scale ($W.m^{-2}$ for instance), or on the transformed log scale (dB) ?