Frequentist Justification for using Student's t-Distribution to Construct Confidence Intervals

Question

In my most recent statistics class, the general¹ frequentist approach to constructing confidence intervals² for an estimator (of any kind) was described as follows:

1. Specify the estimator and the data generating process (i.e., the likelihood $\mathcal{L}(x; \theta)$).
2. Apply the estimator to observed data to obtain the estimate.
3. Calculate the distribution of the estimator, assuming the estimate obtained in Step 2 is the true value of the estimand, and output the appropriate cutoffs to give a (central) $(1 - \alpha)$–confidence-interval.

For instance, in the case of MLE estimators, one would proceed as follows:

1. Construct the likelihood function $\mathcal{L}(x; \theta)$ for the data generating process.
2. Given observed data $X$, find the maximum likelihood estimate $\theta_{\text{MLE}} = \arg\min_{\theta} \mathcal{L}(X; \theta)$.
3. Calculate the distribution of the estimator $\min_{\theta} \mathcal{L}(\tilde X; \theta)$ assuming $\tilde X$ has the distribution implied by $\theta_{\text{MLE}}$—e.g., the density of $\tilde X$ is given by $\mathcal{L}(x; \theta_{\text{MLE}})$—and then find $b_{\text{lower}}$ and $b_{\text{upper}}$ such that $\Pr(\arg\min_{\theta} \mathcal{L}(\tilde X; \theta) < b_{\text{lower}}) = \Pr(\arg\min_{\theta} \mathcal{L}(\tilde X; \theta) > b_{\text{upper}}) = \tfrac \alpha 2$ and output the interval $[b_{\text{lower}}, \, b_{\text{upper}}]$.

This process makes a lot of sense to me as a "theory" of frequentist CIs, but the most basic form of frequentist CI—viz., a CI for a sample mean—doesn't fit with this procedure.

In particular, if we assume that $X = (X_1, \ldots, X_n)$ represents $n$ i.i.d. draws from a normal $\mathcal{N}(\mu, \sigma^2)$, then we have that \begin{align} \mu_{\text{MLE}} &= \tfrac 1 n \sum_{i = 1}^n X_i \\ \sigma_{\text{MLE}} &= \sqrt{\tfrac 1 n \sum_{i = 1}^n (X_i - \mu_{\text{MLE}})^2} \end{align} Therefore, Step 3 suggests that we should look at the distribution of $\sum_{i = 1}^n Y_i$ where $Y_i \sim \mathcal{N}(\mu_{\text{MLE}}, \sigma_{\text{MLE}}^2)$; the distribution of this sample mean is, of course, $\mathcal{N}(\mu_{\text{MLE}}, \tfrac {\sigma_{\text{MLE}}^2} {n})$. However, the standard advice is to construct the confidence interval as if the sample mean were drawn from the $t$-distribution with $n-1$ degrees of freedom (appropriately scaled and recentered). This gives a slightly wider confidence interval which (based on a bunch of simulation I did and a century of received statistical wisdom) generally comes closer to nominal coverage.

Is there any way to make sense of this standard advice in light of the "general theory of frequentist confidence intervals" laid out above? Is this "general theory of frequentist confidence intervals" not actually the general theory at all?

---

1 _{I realize that this is not actually the most general formulation mathematically. I just mean that it was the "most general" in the sense of, "This is how we think about constructing confidence intervals in a frequentist setting."}

2 _{"Frequentist confidence intervals" is, perhaps, tautological if you distinguish between "confidence intervals" and "credible intervals"—I just mean error bars that are not Bayesian (or Fiducial or ...).}

Answer 1

The issue, I've now realized, is that Steps 1, 2, and 3 are not actually the frequentist definition of a confidence interval. According to Wikipedia, the definition is actually a pair of functions $u$ and $v$ such that $\Pr(u(X) < \theta < v(X)) = 1 - \alpha$. Steps 1, 2, and 3 are just a general way of (approximately) constructing such $u$ and $v$. So, the point is, in this particular case, $$ \Pr(\mu_{\text{MLE}} - t^{n-1}_{\alpha/2} \sigma_{\text{MLE}} < \theta) = \tfrac \alpha 2 $$ while $$ \Pr(\mu_{\text{MLE}} - z_{\alpha/2} \sigma_{\text{MLE}} < \theta) > \tfrac \alpha 2 $$ where $t^{n-1}_{\alpha/2}$ is the $\tfrac \alpha 2$-th quantile for a $t$-distribution with $n-1$ degrees of freedom and $z_{\alpha/2}$ is the $\tfrac \alpha 2$-th quantile for a standard normal.