Use Bayes' rule to $p$-value

Question

In hypothesis testing, one usually calculates $p$-value by assuming that null hypothesis holds. In wiki page of $p$-value, first sentence verifies my statement above. To write $p$-value using mathematical notation, I think it is the following $$p-value = P( observation| H_0 \text{ is true }).$$ But in reality, one is interested in its reverse, i.e. $$P( H_0 \text{ is true }| observation ).$$ To connect the 2 quantities, one can use Bayes' rule: $$P(H_0 \text{ is true }| observation) = \frac{P( observation| H_0 \text{ is true }) P(H_0 \text{ is true }) }{P(observation)}.$$ Does my understanding above hold?

I seldom see reasoning above involving $p$-value. So it must be false somewhere in my reasoning. But I could not find it.

Answer 1

The approach that you describe is actually used in a nature publication as an illustration for the (often unknown) complexity involved when interpreting standard p-values. See here: Nuzzo, 2014.

But I agree with Dayne: this approach is not justied by the frequentist paradigm, since in the frequentist sense parameters are not random. If you go Bayesian, then this perfectly legit. However, Bayesians have developed other nice tools that are preferred, like Bayes Factors or posterior predictive p-values.

Answer 2

Good thinking, but here's the problem in your explanation:

"$H_0$ is true" is not a random variable. In the frequentist approach, parameters are (un)known fixed numbers. So the hypothesis is either right or wrong. Therefore the idea of

$P(H_0$ is true $|$ observation$)$ is itself incorrect.

Edit: Based on another answer below, one more thing comes to mind. Even if you consider parameters as random (i.e., Bayesian approach), you can relate $P(H_0$ is true $|$ observation$)$ to MLE. In MLE you maximize something subject to parameters. Basically rather than assuming some value of the parameter in $H_0$, you just assume that distribution comes from a particular family (for example, you may assume distribution is Normal). And then you maximize likelihood function subject to parameters. So an MLE can be though of as finding the $H_0$ for which $P(H_0$ is true $|$ observation$)$ is maximum among various possible hypotheses.