9
$\begingroup$

I am used to knowing "degrees of freedom" as $n - r$, where you have the linear model $$\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}$$ with $\mathbf{y} \in \mathbb{R}^n$, $\mathbf{X} \in M_{n \times p}(\mathbb{R})$ the design matrix with rank $r$, $\boldsymbol{\beta} \in \mathbb{R}^p$, $\boldsymbol{\epsilon} \in \mathbb{R}^n$ with $\boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0}, \sigma^2 \mathbf{I}_n)$, $\sigma^2 > 0$.

From what I recall of elementary statistics (i.e., pre-linear models with linear algebra), the degrees of freedom for the matched-pairs $t$-test is the number of differences minus $1$. So this would entail $\mathbf{X}$ having rank 1, perhaps. Is this correct? If not, why is $n-1$ the degrees of freedom for the matched-pairs $t$-test?

To understand the context, suppose I have a mixed-effects model $$y_{ijk} = \mu_i + \text{ some random effects} + e_{ijk}$$ where $i = 1, 2$, $j = 1, \dots, 8$, and $k = 1, 2$. There is nothing special about $\mu_i$ other than that it's a fixed effect, and $e_{ijk} \overset{iid}{\sim}\mathcal{N}(0, \sigma^2_e)$. I'm assuming that the random effects are irrelevant to this problem, since we only care about the fixed effects in this case.

I would like to provide a confidence interval for $\mu_1 - \mu_2$.

I have already shown that $\bar{d}_\cdot = \dfrac{1}{8}\sum d_j$ is an unbiased estimator of $\mu_1 - \mu_2$, where $d_j = \bar{y}_{1j\cdot} - \bar{y}_{2j\cdot}$, $\bar{y}_{1j\cdot} = \dfrac{1}{2}\sum_{k}y_{1jk}$, and $\bar{y}_{21\cdot}$ is defined similarly. The point estimate $\bar{d}_{\cdot}$ has been computed.

I have already shown that $$s^2_d = \dfrac{\sum_{j}(d_j - \bar{d}_{\cdot})^2}{8-1}$$ is an unbiased estimator of the variance of $d_j$, and thus, $$\sqrt{\dfrac{s^2_d}{8}}$$ is the standard error of $\bar{d}_{\cdot}$. This has been computed.

Now the last part is figuring out the degrees of freedom. For this step, I usually try to find the design matrix - which obviously has rank 2 - but I have the solution to this problem, and it says that the degrees of freedom is $8-1$.

In the context of finding the rank of a design matrix, why are the degrees of freedom $8-1$?

Edited to add: Perhaps helpful in this discussion is how the test statistic is defined. Suppose I have a parameter vector $\boldsymbol{\beta}$. In this case, $$\boldsymbol{\beta} = \begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix}$$ (unless I'm missing something entirely). We are essentially performing the hypothesis test $$\mathbf{c}^{\prime}\boldsymbol{\beta} = 0$$ where $\mathbf{c}^{\prime} = \begin{bmatrix} 1 & -1 \end{bmatrix}$. Then, the test statistic is given by $$t = \dfrac{c^{\prime}\hat{\boldsymbol{\beta}}}{\sqrt{\hat{\sigma}^2c^{\prime}(\mathbf{X}^{\prime}\mathbf{X})^{-1}\mathbf{c}}}$$ which would be tested against a central $t$-distribution with $n - r$ degrees of freedom, where $\mathbf{X}$ is the design matrix as above, and $$\hat{\sigma}^2 = \dfrac{\mathbf{y}^{\prime}(\mathbf{I}-\mathbf{P}_{\mathbf{X}})\mathbf{y}}{n-r}$$ where $\mathbf{P}_{\mathbf{X}} = \mathbf{X}(\mathbf{X}^{\prime}\mathbf{X})^{-1}\mathbf{X}^{\prime}$.

$\endgroup$
5
$\begingroup$

The matched-pairs $t$-test with $n$ pairs is actually just a one-sample $t$-test with a sample of size $n$. You have $n$ differences $d_1,\ldots,d_n$, and these are i.i.d. and normally distributed. $$ \begin{array}{ccccc} \begin{bmatrix} d_1 \\ \vdots \\ d_n \end{bmatrix} & = & \begin{bmatrix} \bar d \\ \vdots \\ \bar d \end{bmatrix} & + & \begin{bmatrix} d_1 - \bar d \\ \vdots \\ d_1 - \bar d \end{bmatrix} \\[10pt] n \text{ d.f.} & & 1 \text{ d.f.} & & (n-1) \text{ d.f.} \end{array} $$ The first column after $\text{“}{=}\text{''}$ has $1$ degree of freedom because of the linear constraint that says all entries are equal; the second has $n-1$ degrees of freedom because of the linear constraint that says the sum of the entries is $0$.

$\endgroup$
  • $\begingroup$ So, in other words, the reason why we have $n-1$ degrees of freedom here has nothing to do with the linear model $\mathbf{y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\epsilon}$? $\endgroup$ – Clarinetist May 1 '16 at 14:15
  • 1
    $\begingroup$ It does have to do with that model, where the matrix $\mathbf X$ is a column of $1$s and $\boldsymbol{\beta}$ is a $1\times1$ matrix whose only entry is the difference between the two population means. $\qquad$ $\endgroup$ – Michael Hardy May 1 '16 at 14:19
  • 2
    $\begingroup$ Aha! So your $\mathbf{y}$ vector would be that vector of $d_i$s, correct? Thank you very much! I can't believe how hard it has been to find an answer on this! $\endgroup$ – Clarinetist May 1 '16 at 14:20
  • $\begingroup$ Yes. It's the vector of observed differences in the $n$ matched pairs. $\qquad$ $\endgroup$ – Michael Hardy May 1 '16 at 14:21
2
$\begingroup$

Many, many thanks to Michael Hardy for answering my question.

The idea is this: let $$\mathbf{y} = \begin{bmatrix} d_1 \\ \vdots \\ d_n \end{bmatrix}$$ and $\boldsymbol{\beta} = [\mu_1 - \mu_2]$. Then our linear model is then $$\mathbf{y} = \mathbf{1}_{n \times 1}\boldsymbol{\beta} + \boldsymbol{\epsilon}$$ where $\mathbf{1}_{n \times 1}$ is the $n$-vector of all ones, and $$\boldsymbol{\epsilon} = \begin{bmatrix} \epsilon_1 \\ \vdots \\ \epsilon_n \end{bmatrix} \sim \mathcal{N}(\mathbf{0}, \sigma^2\mathbf{I}_n)\text{.}$$ Obviously $\mathbf{X} = \mathbf{1}_{n \times 1}$ has rank $1$, so then we have $n-1$ degrees of freedom.

How do we know to set $\boldsymbol{\beta}$ equal to $[\mu_1 - \mu_2]$? Recall that $$\mathbb{E}[\mathbf{y}] = \mathbf{X}\boldsymbol{\beta}$$ and as it can be easily seen, $\mathbb{E}[d_j] = \mu_1 - \mu_2$ for all $j$. Given our $\mathbf{X}$, it is obvious what $\boldsymbol{\beta}$ should be. This is because $$\mathbb{E}[\mathbf{y}] = \mathbb{E}\left[\begin{bmatrix} d_1 \\ \vdots \\ d_n \end{bmatrix} \right] = \begin{bmatrix} \mathbb{E}[d_1] \\ \vdots \\ \mathbb{E}[d_n] \end{bmatrix} = \begin{bmatrix} \mu_1 - \mu_2 \\ \vdots \\ \mu_1 - \mu_2 \end{bmatrix} = \mathbf{X}\boldsymbol\beta = \mathbf{1}_{n \times 1}\boldsymbol\beta = \begin{bmatrix} 1 \\ \vdots \\ 1 \end{bmatrix}\boldsymbol\beta$$ so $\boldsymbol\beta$ should be a $1 \times 1$ matrix with $\boldsymbol\beta = [\mu_1 - \mu_2]$.

Set $\mathbf{c}^{\prime} = [1]$. Then our hypothesis test is $$H_0: \mathbf{c}^{\prime}\boldsymbol{\beta} = 0\text{.}$$ Our test statistic is thus $$\dfrac{\mathbf{c}^{\prime}\hat{\boldsymbol{\beta}}}{\sqrt{\hat{\sigma}^2\mathbf{c}^{\prime}\left(\mathbf{X}^{\prime}\mathbf{X}\right)^{-1}\mathbf{c}}}\text{.}$$ We have $$\hat{\sigma}^2 = \dfrac{\mathbf{y}^{\prime}(\mathbf{I}-\mathbf{P}_{\mathbf{X}})\mathbf{y}}{n-r(\mathbf{X})}\text{.}$$ After some work, it can be shown that $$\mathbf{P}_\mathbf{X} = \mathbf{P}_{\mathbf{1}_{n \times 1}} = \mathbf{1}_{n \times 1}\left(\dfrac{1}{n}\right)\mathbf{1}^{\prime}\text{.}$$ It can also be shown that $\mathbf{I}-\mathbf{P}_{\mathbf{X}}$ is symmetric and idempotent. So, $$\begin{align} \hat{\sigma}^2 &= \dfrac{\mathbf{y}^{\prime}(\mathbf{I}-\mathbf{P}_{\mathbf{X}})\mathbf{y}}{n-r(\mathbf{X})} \\ &= \dfrac{\mathbf{y}^{\prime}(\mathbf{I}-\mathbf{P}_{\mathbf{X}})^{\prime}(\mathbf{I}-\mathbf{P}_{\mathbf{X}})\mathbf{y}}{n-r(\mathbf{X})} \\ &= \dfrac{\|(\mathbf{I}-\mathbf{P}_{\mathbf{X}})\mathbf{y}\|^{2}}{n-r(\mathbf{X})} \\ &=\dfrac{\left\|\left[\mathbf{I}-\mathbf{1}_{n \times 1}\left(\dfrac{1}{n}\right)\mathbf{1}^{\prime}\right]\mathbf{y} \right\|^2}{n-1} \\ &= \dfrac{\left\|\begin{bmatrix} d_1 \\ \vdots \\ d_n \end{bmatrix} - \begin{bmatrix} \bar{d}_\cdot \\ \vdots \\ \bar{d}_\cdot \end{bmatrix} \right\|^2}{n-1} \\ &= \dfrac{\sum_{i=1}^{n}(d_i-\bar{d}_{\cdot})^2}{n-1} \\ &= s^2_d \end{align}$$ and $$\mathbf{X}^{\prime}\mathbf{X} = \mathbf{1}_{n \times 1}^{\prime}\mathbf{1}_{n \times 1} = n$$ which obviously has inverse $1/n$, thus giving a test statistic $$\dfrac{\hat\mu_1-\hat\mu_2}{\sqrt{s^2_d/n}}$$ which would be tested on a central $t$-distribution with $n - 1$ degrees of freedom as desired.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.