I have asked question on CV here so this is a follow up. I believe this time I have the chi-squared test correct.
In this example I'm simulating o
(observed frequency) by increments that deviate from the expected frequency e
so to check at which point alpha=0.05
the observed is equal
to observed values.
It turns out that the observed
values are considered equal at pair
with expected
when the observed differ in % per group
relative to expected by
(results without running the code)
"-2.3552092%" "-1.8722091%" "-1.6773101%"
*Conclussion: *
The above % values are relative % differences between observed and expected (by how much can the observed deviate in % terms from expected at which p.value is > 0.05).
The % results are very tiny amount, the observed values can differ by around 2% only!
*Question: * Is the chi-squared test result not overly sensitive? Meaning small relative % difference is reported being significant when comparing these groups.
*Explanation of the code (for those not using R) *
1) generate seq
of observed values by to point of reaching expected
values per group
2) use this seq
to calc. chi-squared on every combination up to the point where observed
and expected
count matchs 100%.
3) find exact observed
values at which p.value is >0.05.
4) calculate % relative difference of observed
vs. expected
per group. (you get the 3 % values per group as above.
Full simulation of the results below:
x <- data.frame(group=c(1,2,3),
o=c(695301,154100, 224140),
e=c(930785, 192893, 273400))
tb <- xtabs(cbind(o, e) ~ group, data=x)
tb
# visual inspection does not confirm large deviation from expected n. obs.
mosaicplot(tb, col=c(rainbow(10)[2], rainbow(10)[7]))
# create vector nearing the expectation (convergence to expected value
by creating increments)
spl <- with(x, split(x[ ,c("o","e")], group))
lsim <- lapply(spl, function(x) with(x, seq(o, e, ,length.out = 1000)))
dsim <- do.call(cbind, lsim)
o <- dsim; #rownames(o) <- "o"
e <- t(x[ ,"e", drop=FALSE])
simd <- lapply(1:nrow(o), function(i) { oi <- o[i, ,drop=FALSE];
res <- rbind(oi,e); data.frame(group=1:3, t(res)) })
simul <- lapply(simd, function(x) { tb <- xtabs(cbind(V1, e) ~ group,
data=x); chsq <- chisq.test(t(tb));
p.value <- chisq.test(t(tb))$p.value; p.value } )
simres <- round(do.call(rbind, simul),4)
plot(simres ~ 1)
lines(smooth(simres), col="red", lwd=2)
abline(v=which(simres>0.05)[1], col="blue", lwd=2)
min_accept <- simd[[which(simres>0.05)[1]]]; colnames(min_accept)[2] <- "o"
paste0(zapsmall(with(min_accept, 100*(o - e)/e)), "%")
* Final question: *
The provide answer by N. Wang
is correct and confirmed by simulation.
I rescaled the observed and expected by x/100 and run the analysis/simulation. Now the values of % difference is: [1] "-20.690386%" "-16.447257%" "-14.735079%" for p.value >0.05 !
So the question is how to make the test somehow size less
sensitive?
r code
what it does. $\endgroup$