I have a sample $X_1, X_2,\cdots,X_n$ (where $n$ is small), where we believe theoretically that each $X_i$ follows a binomial distribution $B(m,p_i)$, I want to test if all the $p_i$s have the same value $p_0$ (which is known à priori) or not. Is the $\chi^2$ test appropriate in this case?

EDIT: Here's what I want to test:

          square   round      triangle
red      10          10           20
blue     20          60           30
p_∙,j    10/30=0.3 10/70=0.14     20/50=0.4

I want to test whether $p_{\bullet,1}=p_{\bullet,2}=p_{\bullet,3}=0.5$ or not.

  • 1
    $\begingroup$ It depends on the size $m$ of the binomials and on the value of $p_0$. Is $p_0$ an à priori known value? $\endgroup$ – user83346 Mar 31 '17 at 7:32
  • $\begingroup$ @fcop, yes $p_0$ is à priori known. $\endgroup$ – Enthusiastic Mar 31 '17 at 13:20
  • 1
    $\begingroup$ An appropriate test + implementation for the two sample case are given here- stats.stackexchange.com/a/270991/155360. Generalization to more than two samples is straight-forward. $\endgroup$ – user3903581 Mar 31 '17 at 15:28
  • 1
    $\begingroup$ And what is the value of $p_0$ ? And what is $m$? $\endgroup$ – user83346 Mar 31 '17 at 15:30
  • 1
    $\begingroup$ What is the alternative, hypothesis? $H_1: p_{1} = p_{2} =...\neq p_0$? or $H_1: p_{1} \neq p_{2} \neq ... \neq p_0$? $\endgroup$ – user3903581 Mar 31 '17 at 15:38

OK, so here is one way to answer your question. Suppose that you observe: $$ X_1,...,X_i,...,X_k\sim Bin(n_i,\pi_i) $$ and you would like to test for whether: $$ H_0:\pi_1 = \dots = \pi_k = \pi_0. $$ There are several ways that this could be done, but I think that the following might work well for testing whether at least one is zero. For each variable compute a test statistics: $$ T_i = 2\log\frac{\bar{X}_i^{X_i}(1 - \bar{X}_i)^{n_i - X_i}} {\pi_0^{X_i}(1 - \pi_0)^{n_i - X_i}} \sim \chi^{2}_1. $$ For each test statistic compute a p-value according to the $\chi^{2}_1$ distribution to obtain $k$ p-values: $$ p_1,\dots,p_k. $$ Now, apply Benjamini-Hochberg adjustment to the p-values to obtain q-values:(https://en.wikipedia.org/wiki/False_discovery_rate) $$ q_1,...,q_k. $$ A nice thing about the BH adjustment is that it can serve as a global test for whether at least one proportion is different from $\pi_0$. So if: $$ \min_i q_i < \alpha, $$ you can reject the null hypothesis that all propensities are equal to $\pi_0$.

A second option for a global test is to sum the test statistics: $$ \sum_{i=1}^{k} T_i \sim \chi^{2}_k. $$

Here is a toy example for implementation:

manyBinomTest <- function(x, n, p0) {
  props <- x / n
  altlik <- dbinom(x, n, x / n, log = TRUE)
  nulllik <- dbinom(x, n, p0, log = TRUE)
  testStats <- 2 * (altlik - nulllik)
  pvals <- pchisq(testStats, 1, lower.tail = FALSE)
  qvals <- p.adjust(pvals, method = "BH")
  globalTest <- sum(testStats)
  globalPval <- pchisq(globalTest, df = length(x), lower.tail = FALSE)
  return(list(qvals = qvals, global = globalPval))

p0 <- 0.3
n <- 100
k <- 200

reps <- 10^3
qglobal <- numeric(reps)
pglobal <- numeric(reps)
for(i in 1:reps) {
  x <- rbinom(k, n, p0)
  result <- manyBinomTest(x, n, p0)
  qglobal[i] <- min(result$qvals)
  pglobal[i] <- result$global

par(mfrow = c(1, 2))
qqplot(qunif(ppoints(reps)), qglobal,
       main = "test via BH screening")
abline(a = 0, b = 1, col = "red")

qqplot(qunif(ppoints(reps)), pglobal,
       main = "test via likelihood ratio")
abline(a = 0, b = 1, col = "red")

enter image description here


Short answer is yes, it is. However, it will only test the null hypothesis that all parameters are equal. If you are interested which of the $p_i$ are different, you will need to run a post hoc test.

EDIT: OK, I see now where the confusion is. $\chi^2$ tests the goodness of fit, but one of the most common application is testing for dependence. In a way this is exactly the same thing, but commonly one shows it as a contingency table.

We have two variables, $X$ (e.g. "round vs square" or "0 vs 1") and $Y$ (e.g. red vs blue). This gives us $X \times Y$ bins (e.g. four bins: round red, round blue, square red and square blue), which can be neatly represented as a contingency table. Each cell contains the number of events in that cell.

       square      round
red     10          40
blue    20          40

How does this relate to your question? Under independence, we assume that the probabilities of getting a "red" outcome is the same for "squares" and "rounds". We assume that "squares" and "rounds" follow a binomial distribution with the same parameter $p$. This is the null ($H_0$); rejecting it means that there is a dependence between $X$ and $Y$ -- that is, $p$ is different for different $X_i$.

Once again -- "square" and "round" is $X_1$ and $X_2$ from your question, it each follows binomial distribution (with results "red" or "blue") and we want to test, using $\chi^2$, whether the probability of getting "blue" is the same for "squares" as for "rounds".

To do that, we are first proposing a theoretical distribution of the values based on $p_{i,\bullet}$ and $p_{\bullet,j}$ (row-wise and column-wise proportions). In our example,

       square      round   p_i,∙
red     10          40     50/110 = 0.45
blue    20          40     60/110 = 0.55
p_∙,j   30/110=0.27 80/10=0.73

O.45 is the experimental probability of getting a "red" under independence; this is the $p$ from your question, but same for "squares" and "rounds" -- same for all $X_i$ (remember, this is our null!).

Note that the four types of events (four bins) are disjoint and their probabilities obviously add up to 1, just as you have described it for the goodness of fit. Each cell in the contingency table is your $A_{i,j}$, and $\bigcup\limits_{i,j} A_{i,j} = \Theta$, and $A_{i,j} \cap A_{m,n} = \emptyset$ if $i \neq m$ or $j \neq n$.

Assuming independence, the expected numbers of observations in each cell will be $N\cdot p_{\bullet,j} \cdot p_{i,\bullet}$.

This is gives the distribution, and we test how good the data fits it. For this we use $\chi^2$. If we reject the null, we infer that the independence assumption is not fulfilled, which means that some of the $p_i$ corresponding to $X_i$ are different. Precisely what you wanted to test.

  • $\begingroup$ Even if $n$ is small? and shouldn't the sum of probabilities equal to 1 in order to use the chi-squared test? $\endgroup$ – Enthusiastic Mar 30 '17 at 12:17
  • 1
    $\begingroup$ Maybe you should add a specific example. If by $X_i$ you mean a vector of values $\in \{0,1\}$, then why can't you run $\chi^2$ for 3 or 4 groups, just like you run it for two groups? Also, what probabilities do you mean, $p_i$? Why should they add to 1? If 34% of blues are round, and 12% of reds are round, why should 12% + 34% = 100%? $\endgroup$ – January Mar 30 '17 at 14:13
  • $\begingroup$ The chi-squared test was originally made to test the goodness of fit for a multinomial distribution. to use the test for other discrete distributions that take values in $\Theta$, then we need to bin the data such that $\Theta=\bigcup\limits_{i=1}^{k} A_{i}$, where $A_{i} \cap A_{j}= \varnothing$, and if each event $A_i$ has probability $p_i$ then the sum of all probabilities must equal 1. $\endgroup$ – Enthusiastic Mar 30 '17 at 15:40
  • 1
    $\begingroup$ Yeah, but you are confusing $A_i$ with $X_i$. I will try to expand my answer to explain. $\endgroup$ – January Mar 31 '17 at 6:32
  • 1
    $\begingroup$ Ah, I see. The crucial fact was that $p_0$ is known a priori. In that case, generate the expected values for each bin using $p_0$ and test the goodness of fit of your observed values. Also, please note that your $p_{\bullet,j}$ are not the same as mine. $\endgroup$ – January Mar 31 '17 at 14:51

By definition the $\chi^2$ random variable with $n$ degrees of freedom is a sum of $n$ squared independent standard normal variables, i.e. if $X_i \sim N(0,1)$ then $X^2=\sum_{i=1}^n X_i^2$ is $\chi^2(n)$.

It is also known that a binomial variable of size $N$ and success probability $p$ can be approximated by a normal random variable with mean $Np$ and standard deviation $\sqrt{Np(1-p)}$. The approximation works well if p is close to 50% for reasonable sizes $N$.

So if the approximation works well and $B_i \sim Bin(N,p)$ and independent then $X_i=\frac{B_i-Np}{\sqrt{Np(1-p)}}$ is standard normal.

Therefore $\sum_{i=1}^n \left( \frac{B_i-Np}{\sqrt{Np(1-p)}}\right)^2$ is approximately $\chi^2(n)$ and can be used as test statistic in a chi-square test.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.