I've got a set of N normal independent normal distributions, each representing a signal.
I also got a new data sample, a vector $v$ of size Nx1.
Now let's say I compute the p-value using the left-sided z-test of each normal distribution with the respective value in $v$.
$ p_{value} = ztest(v_i, | \mu_i, \sigma_i) $
now, $p_{value}$ will tend to 0 for an unlikely value $v_i$, and to 1 for a more likely value $v_i$.
At this point, I want to threshold all values that are significantly different ($p_{value}<0.01$). However, before doing that, I want to exclude from the threshold those values that comes from "weak" distributions, i.e. distributions that have been built using only few samples of the signal.
For each normal distribution there is associated a prior probability $\pi$, which represents the reliability of the normal distribution (say, the number of valid samples used to build the normal distribution). This prior
- tends to 0 if only few samples of the signal have been used to build it,
- tends to 1 if most of the samples of the signal have been used to build it;
My idea would be $ q_{value} = (1 - p_{value})\\ q_{value} = \pi \times q_{value}\\ p_{value} = 1 - q_{value} $
In this way, the significant p-values that come from a unreliable distribution are excluded from the group of the significant different p-values.
QUESTION This thing works practically. What I would like to know is...does it work theoretically?
Remark the left-sided z-test is defined as the probability of having a value smaller or equal than $x$, given $N(\mu, \sigma)$
