confidence interval for linear regression coefficient with error following a t distribution

Question

I have this linear model $$𝑌=𝛽_0+𝛽_1𝑋+𝜖$$

where the error terms 𝜖 are iid from a student t-distribution with constant degrees of freedom k. I want to construct a 95% confidence interval for $\hat{\beta_1}$. The general formula is ($\hat{\beta_1}-\alpha * se(\hat{\beta_1}), \hat{\beta_1}+\alpha * se(\hat{\beta_1})$). Normally if the error 𝜖 follows a N(0,1) distribution, then $\alpha$ would be the 0.975 quantile for the N(0 1) distribution, which is 1.96. But since 𝜖 follows at distribution here, I am not sure which distribution should I extract the quantile from.

Should I use quantile from the same t distribution where the error is from, or should I use a t distribution with degree of freedom =n-2 (n is the sample size, and minus 2 because I have $\beta_0, \beta_1$ in the model). Can anyone share some thoughts?

Thank you.

Answer 1

I'm going to assume that you still use OLS estimation to fit your model (though you could easily justify departure from that and use the MLE instead in this case). I'm also going to assume that you use a scaled T-distribution for the error, still using the multiplier $\sigma$ to allow the error term variance to have a variable scale. If you don't include this parameter, just take $\sigma = 1$ in my answer and the result is the same.

Assuming you used OLS estimation, your estimated coefficient vector can be written as:

$$\hat{\boldsymbol{\beta}} = \boldsymbol{\beta} + (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \boldsymbol{\epsilon}.$$

We then have:

$$\begin{align} \mathbb{V}(\hat{\boldsymbol{\beta}}) &= [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}] \mathbb{V}(\boldsymbol{\epsilon}) [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}]^\text{T} \\[10pt] &= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \mathbf{I} [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}]^\text{T} \\[6pt] &= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \mathbf{x} (\mathbf{x}^\text{T} \mathbf{x})^{-1} \\[6pt] &= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1}. \\[6pt] \end{align}$$

Therefore, taking $\mathbf{M} \equiv (\mathbf{x}^\text{T} \mathbf{x})^{-1}$ to be the inverse-Gramian matrix in the regression, the true standard error here is:

$$\begin{align} \text{se}_i \equiv \mathbb{S}(\hat{\beta}_i) &= \sqrt{\frac{k}{k-1}} \cdot \sigma \cdot M_{i,i}, \end{align}$$

and the estimated standard error is:

$$\begin{align} \hat{\text{se}}_i \equiv \hat{\mathbb{S}}(\hat{\beta}_i) &= \sqrt{\frac{k}{k-1}} \cdot \hat{\sigma} \cdot M_{i,i}. \end{align}$$

As you can see, the only difference here from the case of a normal distribution is that you have the additional factor $\sqrt{\tfrac{k}{k-2}}$ in the standard error term. If you add this additional factor into the standard error for your confidence interval, it will account for the effect of the degrees-of-freedom parameter on the error variance.