2
$\begingroup$

Suppose I have some i.i.d. normal observations from $\mathbb{R}^f$ with parameters $(\mu, \Sigma)$ and $\Sigma$ is known to be the identity matrix.

I have the following hypotheses: $H_0^i$: $\mu_i = 0$

I calculate the statistics $ T_i = \frac{1}{\sqrt{n}} \sum_j^n x_{ji}$ which are $N(0, 1)$ under $H_0^i$, and the corresponding p-values.

I combine the test statistics/p-values in some way and test the null-hypothesis $H_0 = \bigcap_i H_0^i$.

If I can't reject, I declare $\mu = 0$. If I'm able to reject, I choose the $ T_i$ with the lowest p-value, say $i = 3$, and declare $\mu = [0, 0, \frac{1}{n} \sum_j^n x_{j3}, 0, \dots ]$.

This seems like a bad idea:

etc.

But maybe it's a good idea? https://stats.stackexchange.com/a/207396/142710

I'm reading a famous paper "Unbiased Recursive Partitioning: A Conditional Inference Framework" by Hothorn, Hornik and Zeileis, and if I'm reading it correctly, this is exactly what they're advocating for.

My question is, is it sometimes acceptable to compare p-values for model selection? What is the right way to think about them in this case?

$\endgroup$

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.