i) First, a recommendation:

Use pchisq( -2*sum(log(p-values)), df, lower.tail=FALSE) instead of 1- ... - you're likely to end up with more accuracy for small p-values.

ii) Yes, it's one-sided. Small values of the chi-square statistic indicate that the component p-values tend to be large (that is, a lack of evidence against the overall null). Imagine you were doing a t-test and the sample means were really, really close together... i.e. $|t|$ was unusually small. Would you reject the null hypothesis that they were equal because they were unusually close together?

Clearly not. You might conclude something else was wrong (like one of your assumptions could be faulty, or you used a really bad test, or your calculation might be wrong, or someone fiddled the data, or ...) - but you wouldn't conclude the means were different because they were surprisingly close!

Indeed - what would you do in that situation:

> t.test(x,y,var.equal=TRUE)

    Two Sample t-test

data:  x and y
t = 1e-04, df = 18, p-value = 0.9999
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.7213824  0.7214315
sample estimates:
 mean of x  mean of y 
-0.2161466 -0.2161711 

So there's a two sample t-test with $p$ really close to 1 (~0.999944). What do you conclude?

So now, with a goodness of fit, what kinds of things might a p-value really close to 1 tell you?

i) First, a recommendation:

Use pchisq( -2*sum(log(p-values)), df, lower.tail=FALSE) instead of 1- ... - you're likely to end up with more accuracy for small p-values. To see that they're sometimes going to give different results, try this:

 x=70;c(1-pchisq(x,1),pchisq(x,1,lower.tail=FALSE))

ii) Yes, it's one-sided. Small values of the chi-square statistic indicate that the component p-values tend to be large (that is, a lack of evidence against the overall null). Imagine you were doing a t-test and the sample means were really, really close together... i.e. $|t|$ was unusually small. Would you reject the null hypothesis that they were equal because they were unusually close together?

Clearly not. You might conclude something else was wrong (like one of your assumptions could be faulty, or you used a really bad test, or your calculation might be wrong, or someone fiddled the data, or ...) - but you wouldn't conclude the means were different because they were surprisingly close!

Indeed - what would you do in that situation:

> t.test(x,y,var.equal=TRUE)

    Two Sample t-test

data:  x and y
t = 1e-04, df = 18, p-value = 0.9999
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.7213824  0.7214315
sample estimates:
 mean of x  mean of y 
-0.2161466 -0.2161711 

So there's a two sample t-test with $p$ really close to 1 (~0.999944). What do you conclude?

So now, with a goodness of fit, what kinds of things might a p-value really close to 1 tell you?

i) First, a recommendation:

Use pchisq( -2*sum(log(p-values)), df, lower.tail=FALSE) instead of 1- ... - you're likely to end up with more accuracy for small p-values.

ii) Yes, it's one-sided. Small values of the chi-square statistic indicate that the component p-values tend to be large (that is, a lack of evidence against the overall null). Imagine you were doing a t-test and them sample means were really, really close together... i.e. $|t|$ was unusually small. Would you reject the null hypothesis that they were equal because they were unusually close together?

Clearly not. You might conclude something else was wrong (like one of your assumptions could be faulty, or you used a really bad test, or your calculation might be wrong, or someone fiddled the data, or ...) - but you wouldn't conclude the means were different because they were surprisingly close!

Indeed - what would you do in that situation:

> t.test(x,y,var.equal=TRUE)

    Two Sample t-test

data:  x and y
t = 1e-04, df = 18, p-value = 0.9999
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.7213824  0.7214315
sample estimates:
 mean of x  mean of y 
-0.2161466 -0.2161711 

So there's a two sample t-test with $p$ really close to 1 (~0.999944). What do you conclude?

So now, with a goodness of fit, what kinds of things might a p-value really close to 1 tell you?

i) First, a recommendation:

Use pchisq( -2*sum(log(p-values)), df, lower.tail=FALSE) instead of 1- ... - you're likely to end up with more accuracy for small p-values.

ii) Yes, it's one-sided. Small values of the chi-square statistic indicate that the component p-values tend to be large (that is, a lack of evidence against the overall null). Imagine you were doing a t-test and the sample means were really, really close together... i.e. $|t|$ was unusually small. Would you reject the null hypothesis that they were equal because they were unusually close together?

Clearly not. You might conclude something else was wrong (like one of your assumptions could be faulty, or you used a really bad test, or your calculation might be wrong, or someone fiddled the data, or ...) - but you wouldn't conclude the means were different because they were surprisingly close!

Indeed - what would you do in that situation:

> t.test(x,y,var.equal=TRUE)

    Two Sample t-test

data:  x and y
t = 1e-04, df = 18, p-value = 0.9999
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.7213824  0.7214315
sample estimates:
 mean of x  mean of y 
-0.2161466 -0.2161711 

So there's a two sample t-test with $p$ really close to 1 (~0.999944). What do you conclude?

So now, with a goodness of fit, what kinds of things might a p-value really close to 1 tell you?