Why "sum of squared Pearson residuals" is around "number of dependent variables" in Binomial distribution?

Question

A Pearson residual is defined as: $r_{i}(\theta)=\frac{y_{i}-E(y_{i}|\theta)}{\sqrt{Var(y_{i}|\theta)}}\tag{1}$ Sum of squared standard residuals $X^{2}$ is: $X^{2}=\sum_ir_i^2\tag{2}$ where $y_{i}$ is $i$th random variable ($i$ = 1, 2, ..., N). $\theta$ is also a random variable since it's about Bayesian inference, but this particular question is about "given $\theta$", so I think we can think $\theta$ not "random". N is the number of dependent variables y.

In a binomial model y[i] ~ dbin(theta, n[i]), it is said that

for known $\theta$, we might expect $X^{2}$ to be around N.

For an example in Appendix A, we can expect $X^{2}$ to be 12, given the known posterior $\theta$ = 0.12 (median value). The prior distribution of $\theta$ is uniform. $i$ has the value of 1, 2, 3, ..., 12 because there are twelve hospitals.

The problem is:

I can prove $X^{2}\approx 12$ through simulation in R, as shown in Appendix B. However, I do not know how to prove mathmatically:

$E(X^{2})=N$

Appendix A:

The Table below shows data representing the mortality rates from 12 English hospitals carrarying out heart surgery on children between 1991 and 1995.

Hospital Operations $n_{i}$ Deaths $y_{i}$

Bristol  143 41
Leicester 187 25 
Leeds 323 24
Oxford 122 23
Guys 164 25
Liverpool 405 42
Southampton 239 24
Great Ormond St 482 53
Newcastle 195 26
Harefield 177 25 
Birmingham 581 58
Brompton 301 31

We can use WinBUGS to calculate the posterior distribution of theta, whose median is 0.12.

# WinBUGS code
model {
    for (i in 1:12){
        y[i] ~ dbin(theta, n[i])
    }
    theta ~ dunif(0, 1)
}

# Data:
list(y=c(41,25,24,23,25,42,24,53,26,25,58,31),
     n=c(143,187,323,122,164,405,239,482,195,177,581,301))

Appendix B:

R codes are used to prove that in the example in Appendix A, for known $\theta$ (i.e., 0.12) we might expect sum of squared standard residuals to be total number of hospitals (i.e., 12).

# number of hospital
N_hosp <- 12  
# theta's value is from posterior mode to 
# check what "we might expect" for known theta
theta <- 0.12 
# number of operations in ith hospital
n=c(143,187,323,122,164,405,239,482,195,177,581,301)
# number of observed deaths in ith hospital
y=c(41,25,24,23,25,42,24,53,26,25,58,31)
# expected y for known theta
y_expect <- array(dim = c(N_hosp))
# Pearson residual
res <- array(dim = c(N_hosp))
res2 <- array(dim = c(N_hosp))

# sample from Binomial to simulate deaths that "we might expect"
# calculate sum of squared stand. residuals
cal_SRes2 <- function(){
  for (i in 1:N_hosp) {
  y_expect[i] <- rbinom(1,n[i],theta)
  res[i] <- (y_expect[i] - n[i]*theta)/sqrt(n[i]*theta*(1-theta))
  res2[i] <- res[i]*res[i]
}
# sum of squared residuals
  sum(res2)
}

# run 50000 times to see if "sum(res2)" is around 12
tmp <- 0
for (j in 1:50000) {
  tmp <- tmp + cal_SRes2()
}
avg_sum_resids <- (1/50000)*tmp  # it is 12, proved

Answer 1

Let $Y_i\mid\theta\sim G_\theta$, for $i=1,\dots,N$ (in the original problem statement, $G_\theta$ stands for a $\text{Bin}(n_i,\theta)$ distribution, but this simple result is more general), and suppose that $\mathbb{E}[Y_i^2\mid\theta]<\infty$. Defining $$ R_i := \frac{Y_i - \mathbb{E}[Y_i\mid\theta]}{\sqrt{\text{Var}[Y_i\mid\theta]}}, $$ it follows immediately (can you see why?) that $\mathbb{E}[R_i\mid\theta]=0$ and $\text{Var}[R_i\mid\theta]=1$. Hence (why?), $$ \mathbb{E}[R_i^2\mid\theta]=\mathbb{E}^2[R_i\mid\theta]+\text{Var}[R_i\mid\theta]=1, $$ and defining $X^2 := \sum_{i=1}^N R_i^2$, we have that $\mathbb{E}[X^2\mid\theta] = \sum_{i=1}^N \mathbb{E}[R_i^2\mid\theta]=N$ (linearity of the expectation).

Answer 2

• Answer 2019/07/17

We need to prove $E(X^{2})=N$, which is $E[\sum_{i=1}^Nr_i^2]=N$.

So the question becomes to prove the following equation: $E[r_i^2]=1\tag{1}$ for each $i$.

Proof process:

Because \begin{equation} \begin{aligned} E(r_i^2)&=E(\frac{y_{i}-E(y_{i}|\theta)}{\sqrt{Var(y_{i}|\theta)}}*\frac{y_{i}-E(y_{i}|\theta)}{\sqrt{Var(y_{i}|\theta)}})\\&=E(\frac{(y_{i}-E(y_{i}|\theta))^{2}}{Var(y_{i}|\theta)}) \end{aligned} \end{equation}

and \begin{equation} \begin{aligned} E((y_{i}-E(y_{i}|\theta))^{2})=Var(y_{i}|\theta) \end{aligned} \end{equation}

So \begin{equation} \begin{aligned} E(r_i^2)=1 \end{aligned} \end{equation}

• Answer 2019/07/16

When considered as a function of random $y_{i}$ for fixed $\theta$, then $r_{i}(\theta)$ has mean 0 and variance 1. Addtionally, if we suppose $r_{i}(\theta)$ are mutually independent standard normal random variables (i.e., I add an extra requirement that $r_{i}(\theta)$ is distributed as Normal), $X^{2}=\sum_ir_i^2$ will has a Chi-square distribution. Since i = 1, 2,...,12 in the example, the mean of Chi-square distributed $X^{2}$ is 12.

However, $r_{i}(\theta)$ is not necessarily "normal" distributed, and I was wondering that I didn't use the Central Limit Theorem when proving.