Estimating parameters of Student's t-distribution

Question

What are the maximum-likelihood estimators for the parameters of Student's t-distribution? Do they exist in closed form? A quick Google search didn't give me any results.

Today I am interested in the univariate case, but probably I will have to extend the model to multiple dimensions.

EDIT: I am actually mostly interested in the location and scale parameters. For now I can assume that the degrees of freedom parameter is fixed, and possibly use some numeric scheme to find the optimal value later.

Answer 1

Closed form does not exist for T, but a very intuitive and stable approach is via the EM algorithm. Now because student is a scale mixture of normals, you can write your model as

$$y_i=\mu+e_i$$

where $e_i|\sigma,w_i \sim N(0,\sigma^2w_i^{-1})$ and $w_i\sim Ga(\frac{\nu}{2}, \frac{\nu}{2})$. This means that conditionally on $w_i$ the mle are just the weighted mean and standard deviation. This is the "M"step

$$\hat{\mu}=\frac{\sum_iw_iy_i}{ \sum_iw_i}$$ $$\hat{\sigma}^2= \frac{\sum_iw_i(y_i-\hat{\mu})^2}{n}$$

Now the "E" step replaces $w_i$ with its expectation given all the data. This is given as:

$$\hat{w}_i=\frac{(\nu+1) \sigma^2 }{\nu \sigma^2 +(y_i-\mu)^2}$$

so you simply iterate the above two steps, replacing the "right hand side" of each equation with the current parameter estimates.

This very easily shows the robustness properties of the t distribution as observations with large residuals receive less weight in the calculation for the location $\mu$, and bounded influence in the calculation of $\sigma^2$. By "bounded influence" I mean that the contribution to the estimate for $\sigma^2$ from the ith observation cannot exceed a given threshold (this is $(\nu+1)\sigma^2_{old}$ in the EM algorithm). Also $\nu$ is a "robustness"parameter in that increasing (decreasing) $\nu$ will result in more (less) uniform weights and hence more (less) sensitivity to outliers.

One thing to note is that the log likelihood function may have more than one stationary point, so the EM algorithm may converge to a local mode instead of a global mode. The local modes are likely to be found when the location parameter is started too close to an outlier. So starting at the median is a good way to avoid this.

Answer 2

The following paper addresses exactly the problem you posted.

Liu C. and Rubin D.B. 1995. "ML estimation of the t distribution using EM and its extensions, ECM and ECME." Statistica Sinica 5:19–39.

It provides a general multivariate t-distribution parameter estimation, with or without the knowledge of the degree of freedom. The procedure can be found in Section 4, and it is very similar to probabilityislogic's for 1-dimension.

Answer 3

I doubt that it exists in closed form: if you write any one of the factors of the likelihood as $$\frac{\Gamma(\frac{\nu+1}{2})} {\sqrt{\nu\pi}\,\Gamma(\frac{\nu}{2})} \left(1+\frac{t^2}{\nu} \right)^{-\frac{\nu+1}{2}} = \frac{\Gamma(\frac{\nu+1}{2})} {\sqrt{\nu\pi}\,\Gamma(\frac{\nu}{2})} \exp \left \{ \left [ \ln \left(1+\frac{t^2}{\nu} \right) \right ] \left [ {-\frac{\nu+1}{2}} \right ]\right \}$$ and take the ln of that, you will get a nonlinear equation in $\nu$. Even if you manage to get a solution, then depending on the number of factors (terms) $n$, the MLE equation is going to depend on this $n$ in a nontrivial way. All that dramatically simplifies, of course, when $\nu \rightarrow \infty$, when the power approaches an exponential (Gaussian PDF).

Answer 4

Does a closed-form maximum-likelihood estimator for the Student's t distribution exist? The answer is now YES!! During the COVID pandemic, I dug into this problem and discovered a method I call Independent Approximators (IAs). This new algorithm provides a closed-form estimate of the location, scale, and shape that achieves the maximum likelihood estimate. The method works by filtering the samples by pairs and triplets that are approximately equal. The IA-pairs are distributed as the normalized square of the original distribution and are guaranteed to have a defined mean. The IA-triplets are distributed as the normalized cube of the original distribution, are guaranteed to have a finite second moment, and are used to estimate the scale. Finally, the geometric mean is used to estimate the scale, as defined in a paper I posted earlier to the question.

Try it out. I'm quite interested in getting feedback on this new method. Mathematica code is available in the referenced Github repository.

Kenric Nelson, "Independent Approximates enable closed-form estimation of heavy-tailed distributions"

https://arxiv.org/abs/2012.11026

---

Original answer from 2018:

I have recently discovered a closed-form estimator for the scale of the Student's t distribution. To the best of my knowledge, this is a new contribution, but I would welcome comments suggesting any related results. The paper describes the method in the context of a family of "coupled exponential" distributions. The Student's t is referred to as the Coupled Gaussian, where the coupling term is the reciprocal of the degree of freedom. The closed-form statistic is the geometric mean of the samples. Assuming a value of the coupling or degree of freedom, an estimate of the scale is determined by multiplying the geometric mean of the samples by a function involving the coupling and a harmonic number.

Use of the geometric mean as a statistic for the scale of the coupled Gaussian distributions, Kenric P. Nelson, Mark A. Kon, Sabir R. Umarov