Example 2.2.2 fromAn Introduction to Generalized Linear Models. Show $Y_{j k}, \bar{Y}_{j}, \hat{\beta}_{j} $ are all independent

Question

This question is related to what I have asked in my previous post: How to derive the covariance matrix between $\bar{y}$ and $\hat{\beta_c}$ where $\hat{\beta_c}$ is the OLS estimator of a linear model?.

The example is from the book An Introduction to Generalized Linear Models by Annette J. Dobson, Adrian Barnett(2008)

The motivating example is about birthweight and gestational age.

A fairly general model relating birthweight to gestational age is

$$\mathrm{E}\left(Y_{j k}\right)=\mu_{j k}=\alpha_{j}+\beta_{j} x_{jk}$$ where $x_{j k}$ is the gestational age of the $k$ th baby in group $j$. The intercept parameters $\alpha_{1}$ and $\alpha_{2}$ are likely to differ because, on average, the boys were heavier than the girls. The slope parameters $\beta_{1}$ and $\beta_{2}$ represent the average increases in birthweight for each additional week of gestational age. The question of interest can be formulated in terms of testing the null hypothesis $\mathrm{H}_{0}$ :$\beta_{1}=\beta_{2}=\beta($ that is, the growth rates are equal and so the lines are parallel) against the alternative hypothesis $\mathrm{H}_{1}: \beta_{1} \neq \beta_{2}$ We can test $\mathrm{H}_{0}$ against $\mathrm{H}_{1}$ by fitting two models $$ \begin{array}{l} \mathrm{E}\left(Y_{j k}\right)=\mu_{j k}=\alpha_{j}+\beta x_{j k} ; \quad Y_{j k} \sim \mathrm{N}\left(\mu_{j k}, \sigma^{2}\right) \\ \mathrm{E}\left(Y_{j k}\right)=\mu_{j k}=\alpha_{j}+\beta_{j} x_{j k} ; \quad Y_{j k} \sim \mathrm{N}\left(\mu_{j k}, \sigma^{2}\right) \end{array} $$

Then later the book suggests:

$$ \begin{aligned} Y_{j k} & \sim \mathrm{N}\left(\alpha_{j}+\beta_{j} x_{j k}, \sigma^{2}\right) \\ \bar{Y}_{j} & \sim \mathrm{N}\left(\alpha_{j}+\beta_{j} \bar{x}_{j}, \sigma^{2} / K\right) \\ b_{j} & \sim \mathrm{N}\left(\beta_{j}, \sigma^{2} /\left(\sum_{k=1}^{K} x_{j k}^{2}-K \bar{x}_{j}^{2}\right)\right) \end{aligned} $$

and claimed they are all independent.

The formula for $b_j$ is provided:

$$ b_{j}=\frac{K \sum_{k} x_{j k} y_{j k}-\left(\sum_{k} x_{j k}\right)\left(\sum_{k} y_{j k}\right)}{K \sum_{k} x_{j k}^{2}-\left(\sum_{k} x_{j k}\right)^{2}} $$

I at first thought the proof will be easily carried out using matrix formation at first, which is why I asked my previous post. However, later I realized I may use other properties of covariance operation to finish the proof.

---

Below shows my sketch of how I demonstrate $\hat{\beta_j}$ are independent to $\bar{Y}_{ij}, \bar{y}_j$:

Since we have assumed normal distribution of the normal term, no contrarians between the estimator indicate they are independent.

Using the fact that $\operatorname{cov}(a X, b Y)=(a b) \operatorname{cov}(X, Y)$ : If we look at $\operatorname{cov}\left(\hat{\beta}, Y_{j k}\right)$ from the model assumption we know, $\operatorname{cov}\left(Y_{j k}, Y_{j k}\right)=$ $\operatorname{var}\left(Y_{j k}\right),$ as samples are i.i.d. the covariance of random variable goes to $0$.

Basically, if you look at $\hat{\beta}_{j}=\frac{K\left(\sum_{k} x_{j k} y_{j k}\right)-\left(\sum_{k} x_{j k}\right)\left(\sum_{k} y_{j k}\right)}{K \sum_{k} x_{j k}^{2}-\left(\sum_{k} x_{j k}\right)^{2}},$ a random variable. The denominator is a constant $(\equiv A)$ therefore can be extracted out from the Covariance operation. Finally, the operation will reduce to form: $$\operatorname{cov}\left(\hat{\beta}, Y_{i j}\right)=\frac{K x_{j k}-K x_{j k}}{A}\times\sigma^{2}=0$$

Also, $\bar{Y}_{j}=\frac{\sum_{k} y_{j k}}{K},$ if $\hat{\beta}$ is independent from $Y_{i j},$ so will it be independent from $\bar{Y}_{j}$ as $\operatorname{cov}\left(\hat{\beta}, \bar{Y}_{j}\right)=\sum_{k} \operatorname{cov}\left(\hat{\beta}, Y_{i j}\right) / k=0$

---

Notice that the expression for $\bar{Y}_j$ is not given and I think $$ \bar{Y}_{j}=\frac{\sum_{k} y_{j k}}{K} $$should be the correct way to express it. However, using this expression I am unable to show independence between $\bar{Y}_j$ and $Y_{ij}$. The covariance of the two won't go to 0. However, I feel my expression has some problems as it seems I am getting the covariance conditional on the sex. On the other hand, intuitively it doesn't make sense a group average is uncorrelated with its observation... I am unable to decern where exactly is my problem and fix it. Could someone please point out to me the correct way to demonstrate the independence of these three random variables?

Answer 1

I'd say that (a) a self-study tag is missing, because that claim actually is an exercise :) (b) you should take another path.

Dobson (2nd ed.) & Barnett (3rd ed.) say: "Using the results of Exercise 1.4 show that..." So my hint is based on that exercise.

In general, $\sum(y_i-\bar{y})^2=\sum y_2^2-2n\bar{y}$. Replacing $y_i$ with $y_i-\mu$, whose mean is $\bar{y}-\mu$, we get: \begin{align*} (n-1)S^2=\sum(y_i-\bar{y})^2&=\sum((y_i-\mu)-(\bar{y}-\mu))^2\\ &=\sum(y_i-\mu)^2-n(\bar{y}-\mu)^2 \end{align*} hence, \begin{align*} \sum(y_i-\mu)^2&=\sum(y_i-\bar{y})^2+n(\bar{y}-\mu)^2 \end{align*} Dividing by $\sigma^2$: \begin{align*} \frac{\sum(y_i-\mu)^2}{\sigma^2}&=\frac{(n-1)S^2}{\sigma^2}+\frac{n(\bar{y}-\mu)^2}{\sigma^2} \end{align*} Now: \begin{align*} \frac{\sum(y_i-\mu)^2}{\sigma^2}&\sim\chi^2_n \\ \frac{(n-1)S^2}{\sigma^2}&\sim\chi^2_{n-1}\\ \frac{n(\bar{y}-\mu)^2}{\sigma^2}&\sim\chi^2_1 \end{align*} A chi squared r.v. is the sum of independent standard normal variables, and the sum of $X_1\sim\chi^2_h$ and $X_2\sim\chi^2_k$ is distributed as a $\chi^2_{h+k}$ r.v. only if $X_1$ and $X_2$ are independent. Therefore $S^2$ and $\bar{y}$ are independent random variables.

What will happen if we replace $\mu$ with $\alpha+\beta x$ and $\bar{y}$ with $a+bx$?

HTH