2
$\begingroup$

I have this linear model $$π‘Œ=𝛽_0+𝛽_1𝑋+πœ–$$

where the error terms πœ– are iid from a student t-distribution with constant degrees of freedom k. I want to construct a 95% confidence interval for $\hat{\beta_1}$. The general formula is ($\hat{\beta_1}-\alpha * se(\hat{\beta_1}), \hat{\beta_1}+\alpha * se(\hat{\beta_1})$). Normally if the error πœ– follows a N(0,1) distribution, then $\alpha$ would be the 0.975 quantile for the N(0 1) distribution, which is 1.96. But since πœ– follows at distribution here, I am not sure which distribution should I extract the quantile from.

Should I use quantile from the same t distribution where the error is from, or should I use a t distribution with degree of freedom =n-2 (n is the sample size, and minus 2 because I have $\beta_0, \beta_1$ in the model). Can anyone share some thoughts?

Thank you.

$\endgroup$
7
  • $\begingroup$ The quantile depends only on the confidence interval's "width" that you prefer. If it's a 95% confidence interval, why should your quantile be different. It will be the same: 0.975. $\endgroup$
    – Alex
    Sep 27, 2022 at 22:27
  • $\begingroup$ sorry I meant to say which distribution should I extract the quantile from $\endgroup$
    – AlexH
    Sep 27, 2022 at 22:56
  • $\begingroup$ you said you wanted to extract it from the t-distribution, don't you. $\endgroup$
    – Alex
    Sep 27, 2022 at 22:57
  • $\begingroup$ yeah, but I am not sure which degree of freedom should I use? Should I use k or n-2? $\endgroup$
    – AlexH
    Sep 27, 2022 at 23:07
  • $\begingroup$ I don't know. The t-distribution was your assumption. What degree of freedom do you assume for your error term? $\endgroup$
    – Alex
    Sep 27, 2022 at 23:11

1 Answer 1

1
$\begingroup$

I'm going to assume that you still use OLS estimation to fit your model (though you could easily justify departure from that and use the MLE instead in this case). I'm also going to assume that you use a scaled T-distribution for the error, still using the multiplier $\sigma$ to allow the error term variance to have a variable scale. If you don't include this parameter, just take $\sigma = 1$ in my answer and the result is the same.

Assuming you used OLS estimation, your estimated coefficient vector can be written as:

$$\hat{\boldsymbol{\beta}} = \boldsymbol{\beta} + (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \boldsymbol{\epsilon}.$$

We then have:

$$\begin{align} \mathbb{V}(\hat{\boldsymbol{\beta}}) &= [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}] \mathbb{V}(\boldsymbol{\epsilon}) [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}]^\text{T} \\[10pt] &= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \mathbf{I} [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}]^\text{T} \\[6pt] &= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \mathbf{x} (\mathbf{x}^\text{T} \mathbf{x})^{-1} \\[6pt] &= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1}. \\[6pt] \end{align}$$

Therefore, taking $\mathbf{M} \equiv (\mathbf{x}^\text{T} \mathbf{x})^{-1}$ to be the inverse-Gramian matrix in the regression, the true standard error here is:

$$\begin{align} \text{se}_i \equiv \mathbb{S}(\hat{\beta}_i) &= \sqrt{\frac{k}{k-1}} \cdot \sigma \cdot M_{i,i}, \end{align}$$

and the estimated standard error is:

$$\begin{align} \hat{\text{se}}_i \equiv \hat{\mathbb{S}}(\hat{\beta}_i) &= \sqrt{\frac{k}{k-1}} \cdot \hat{\sigma} \cdot M_{i,i}. \end{align}$$

As you can see, the only difference here from the case of a normal distribution is that you have the additional factor $\sqrt{\tfrac{k}{k-2}}$ in the standard error term. If you add this additional factor into the standard error for your confidence interval, it will account for the effect of the degrees-of-freedom parameter on the error variance.

$\endgroup$
3
  • $\begingroup$ +1 But then what happens if $k\le2?$ $\endgroup$
    – Dave
    Sep 27, 2022 at 23:52
  • $\begingroup$ In that case the error variance is infinite, so the estimator standard error is also infinite. $\endgroup$
    – Ben
    Sep 28, 2022 at 4:02
  • $\begingroup$ The underlying question concerns whether this formula attains the desired coverage. For smallish datasets and smaller degrees of freedom, I (and, I believe, the OP) suspect it might not, due to the heavy tails of the error variables. $\endgroup$
    – whuber
    Sep 28, 2022 at 13:53

Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.