Recently we've been learning hypothesis testing in class, and I have an idea about incorporating probability integral transformation into it, which I think have some decent advantages over the current methodology. Let me explain it with an example:

Consider a trivial one-sample $z$-test, which has $H_0: \mu \leq \mu_0$ versus $H_1: \mu > \mu_0$. If we take the conventional path, a good test statistic would be

$$z = \frac{\bar x - \mu_0}{\sigma / \sqrt{n}}$$

with the region of rejection being

$$z \geq z_{1-\alpha}$$

Now, my idea is to use the following test statistic

$$\mathbb{z} = \Phi \left( \frac{\bar x - \mu_0}{\sigma / \sqrt{n}} \right)$$

where $\Phi$ is the CDF of the standard normal distribution (which is the distribution of $z$). Then the region of rejection will become

$$\mathbb{z} \geq 1-\alpha$$

This is because $\mathbb{z}$ has a uniform distribution on the interval $[0, 1]$.

Basically, I'm feeding whatever test statistic into the CDF of its distribution, and the resulting test statistic would have a standard uniform distribution (here is why). In my humble opinion, this method has several advantages:

  • The region of rejection in most hypothesis testing problems (including $z$-test, $t$-test, $\chi^2$ test and $F$-test, among others) would be similar, i.e. $[0, \alpha]$, $[1-\alpha, 1]$, or $[0, \frac{\alpha}{2}] \cup [1-\frac{\alpha}{2}, 1]$.
  • The calculations of $p$-values are also greatly simplified. For example, it's simply $1 - \mathbb{z}$ in this case.
  • Since CDFs are non-decreasing, the "shape" of the region of rejection won't change.
  • With modern technologies like MATLAB or R, calculating $\mathbb{z}$ is as convenient as, if not even more convenient than, finding $z_{1-\alpha}$.

I know this is by no means a huge improvement over the current implementation, but I think it can make hypothesis testing easier. Someone must have thought about the same method before I do, so why isn't it widely used?

  • $\begingroup$ Well, as you point out, that is tantamount to using p-values, so it is quite widely used. $\endgroup$ – Christoph Hanck May 14 '18 at 9:40

I could be mistaken, but it looks to me like you're literally describing the procedure for calculating the p-value corresponding to a z-statistic (your new $z$ being equal to $1-p$). I need hardly point out that p-values are indeed widely used.

Transforming test statistics (e.g. $F$, $t$, $\chi^2$, etc.) to p-values is useful because it renders them into a common space with an intuitive interpretation, from which you can directly draw inferences about your hypotheses. But it's also useful to retain and report the original statistic, because it has a bit more information about how you arrived at your conclusion, and thus is more transparent to your audience.

  • $\begingroup$ so I reinvented the $p$-value? sounds cool :) $\endgroup$ – nalzok May 14 '18 at 11:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.