# Questions tagged [combining-p-values]

Methods for combining p-values from several hypothesis tests with the same null hypothesis into one aggregate p-value, such as e.g. Fisher's method and Stouffer's method.

71 questions
Filter by
Sorted by
Tagged with
25 views

### Test for change in all observerd variables

Let us say, I observe 3 variables in a control condition and a treatment condition. I would like to find out whether the treatment has some effect on all 3 variables at the same time. I should mention ...
46 views

### How to summarize multiple independent false discovery rates (FDRs or q-values)?

In the context of high-throughput biological assays (e.g. genomics), where we are applying a hypothesis test (e.g. differential expression between groups of samples) to each analyte in a dataset with ...
26 views

### Sequential hypothesis test for uniform distribution

Suppose I have a sequence of data points $x_1,x_2,\dots$. I want to do a sequential hypothesis test, were the null hypothesis is that each $x_i$ is iid uniform on $[0,1]$, and the alternative ...
81 views

### How to interpret the harmonic mean p-value?

I am working on a multiple comparisons problem for several hypothesis tests and have come across the harmonic mean $p$-value (HMP) for combining the $p$-value of these tests. Specifically, I am using ...
35 views

### Combining tests: Maximum test statistic method but one test is a permutation test

I want to combine two tests $A,B$ for the same null and alternative hypotheses. The motivation is that it has been shown that test $A$ has higher power in some situations whereas test $B$ has higher ...
61 views

### Combining p-values from permutation tests from single subjects

I have a question that's very similar to this question: Combine multiple independent permutation tests My 'issue' is exactly the same: I have 20 subjects and I used permutation tests to determine the ...
67 views

### combined p values from different hypothesis

I am currently working on a project in which I'm simultaneously identifying and quantifying protein in k samples. Experimentally, for each of the k samples, I have n identified proteins, with a p ...
19 views

91 views

### Test for multiple samples of ordinal data

I have 10 independent samples of ordinal data ranked 1-3. I also have predicted ranks for the same data set. I want to test if the predicted ranking matches the observed ranking at a rate better ...
37 views

### Marginalizing statistical test results in a two-factor grid

Consider the following toy problem. Suppose that we have several food groups: meat, vegetables, fruit, etc. For any given pair of food types A and B, we ask whether people that eat both A AND B have a ...
71 views

### Interim analysis: probability of stopping

Say we do some time-varying study. We check yearly if early stopping is necessary, which is done via interim analysis (O'Brien-Fleming stopping boundaries). The 4 p-values for stopping in 4 ...
48 views

### How to test the significant of multiply chi square tests

I know that if I have a model from which I have a very large sample, so that I can create a null distribution. And I also have an experimental sample, I can use the ChiSquare test (or FisherExact test)...
51 views

### Combining the conclusions from multiple Diebold-Mariano tests

Suppose I have two forecasting models, A and B. Moreover, suppose I have multiple datasets and I use each of these sets to perform Diebold-Mariano tests. My aim is to find out which of the two models ...
412 views

### How to combine multiple p-values into one? [closed]

I have the data of 841 genes along with the p-values which represents the association of that gene in a particular tissue (Aorta in this case) with some diseased phenotype. I want to combine the p-...
3k views

### Does a uniform distribution of many p-values give statistical evidence that H0 is true?

A single statistical test can give evidence that the null hypothesis (H0) is false and therefore the alternative hypothesis (H1) is true. But it cannot be used to show that H0 is true because failure ...
546 views

### Bonferroni-type corrections and Fisher Method Meta-analysis

I'm not an expert on the field, but I need to know more about how to combined multiple significance test p-values after a two sample Kolmogorov Smirnov (KS)test on two distributions. But I have two ...
2k views

### Problems with Fisher's Method for combining p-values

I am using Fisher's Method to combine p-values, and have noticed some strange behavior for large p-values and large $n.$ In my case I have a large number of not statistically significant results(e.g. ...
136 views

### Rolling sum of 2 sample KS test results

In order to compare two lists of samples from 'before-treatment' and 'after-treatment', I am doing a two sample KS test using the ks_2samp function from Python's scipy.stats package which gives me the ...
1k views

### GLMM with highly unbalanced data and statistical power

Although I have used R to explain my question, I think and hope this is of general interest and is not off-topic here. I want to test whether the continuous variable “varA” is different between the ...
1k views

### How to achieve a two-sided combined p-value using Fisher's method?

Let's say I want to answer the question if smokers vs nonsmokers have different levels of Gene A. This seems like an obvious two-sided test. However, if I have multiple studies and I want to combine ...
115 views

### Comparing 2 p values of different sample size

I did 2 linear regression analyses. White woman (WW) N1= 50 Black woman (BW) N2= 49 Correlation btw WW's V1 & V2 P = 0.001 Correlation btw BW's V1 & V2 P = 0.013 How do i know if the 2 p ...
336 views

### Probability density Function for p-value product

I am running an experiment where I get 22,152,480 p-values in a given order, ranging from 0 to 1, but strongly tilted toward 1 (i.e. not uniformly distributed). I want to say that a given series of ...
168 views

### Combining p values: what can we say about population of studies

I have N p values (N is around 200) one for each of N independent experiments in my study. These p values range from 0.03 to 0.7 (mean is 0.3), where only 5% of p values are less than 0.05. If I look ...
4k views

### Can a meta-analysis of studies which are all "not statistically signficant" lead to a "significant" conclusion?

A meta-analysis includes a bunch of studies, all of which reported a P value greater than 0.05. Is it possible for the overall meta-analysis to report a P value less than 0.05? Under what ...
285 views

### Performing a meta-analysis of wilcoxon rank sum analyses

Suppose an experiment is done and the following is presented in a paper: median for experimental, median for control, and a WRS p-value, and (n) for EXP and CTRL arms. Now suppose experiment 2 is ...
259 views

### Suppose I have a Cramér-von Mises $p = 0.99$ and Chi-squared $p = 0.88$ of a distribution being Student's-$t$. What can I say?

Suppose I have a Cramér-von Mises $p = 0.99$ and Chi-squared $p = 0.88$ of a distribution being Student's-$t$. What can I say? What can I say about the underlying distribution's being a Student's-$t$...
70 views

### How would you compare subgroup Stouffer Zs?

p-curve analyses (http://www.p-curve.com/) use significant p-values to estimate evidential value and effect sizes. To estimate evidential value, p-values are computed for p-values (pp-values), and ...
48 views

### Can one use Stouffer's method to integrate t-values? [duplicate]

I would like to integrate T-statistics values from multiple, independent tests. How can this be done? Since my degrees of freedom are many ($n>1000$) I was thinking to use the Stouffer's method,...
138 views

### Using p-values reported as inequalities such as $p<0.05$ in meta-analysis; can I convert them to $0.05$?

I am currently undertaking a meta-analysis of the expression profile of specific biomarkers in a specific disease. During the extraction process, I extracted the name of the biomarkers and their ...
429 views

### Testing multiple comparisons of paired proportions

I want to know if a proportion increased significantly after an event, and then find out if that holds true over multiple events. I'm using a paired t-test to measure this with continuous data (e.g. ...
419 views