# Estimating parameters of Student's t-distribution

What are the maximum-likelihood estimators for the parameters of Student's t-distribution? Do they exist in closed form? A quick Google search didn't give me any results.

Today I am interested in the univariate case, but probably I will have to extend the model to multiple dimensions.

EDIT: I am actually mostly interested in the location and scale parameters. For now I can assume that the degrees of freedom parameter is fixed, and possibly use some numeric scheme to find the optimal value later.

• To my knowledge they do not exist in closed form. A gradient ascent type approach may be required.
– Pat
Jul 8, 2013 at 16:30
• Although the Student t distribution has a single parameter, you refer to "parameters" in the plural. Are you perhaps including location and/or scale parameters?
– whuber
Jul 8, 2013 at 19:42
• @whuber, thanks for the comment, I am indeed interested in location and scale parameters, more than in the degrees of freedom. Jul 9, 2013 at 9:51
• With $n$ data, the likelihood equation for the location parameter is algebraically equivalent to a polynomial of degree $2n-1$. Do you consider a zero of such a polynomial to be given in "closed form"?
– whuber
Jul 9, 2013 at 13:23
• Only for $n=2$ :-).
– whuber
Jul 10, 2013 at 13:43

Closed form does not exist for T, but a very intuitive and stable approach is via the EM algorithm. Now because student is a scale mixture of normals, you can write your model as

$$y_i=\mu+e_i$$

where $e_i|\sigma,w_i \sim N(0,\sigma^2w_i^{-1})$ and $w_i\sim Ga(\frac{\nu}{2}, \frac{\nu}{2})$. This means that conditionally on $w_i$ the mle are just the weighted mean and standard deviation. This is the "M"step

$$\hat{\mu}=\frac{\sum_iw_iy_i}{ \sum_iw_i}$$ $$\hat{\sigma}^2= \frac{\sum_iw_i(y_i-\hat{\mu})^2}{n}$$

Now the "E" step replaces $w_i$ with its expectation given all the data. This is given as:

$$\hat{w}_i=\frac{(\nu+1) \sigma^2 }{\nu \sigma^2 +(y_i-\mu)^2}$$

so you simply iterate the above two steps, replacing the "right hand side" of each equation with the current parameter estimates.

This very easily shows the robustness properties of the t distribution as observations with large residuals receive less weight in the calculation for the location $\mu$, and bounded influence in the calculation of $\sigma^2$. By "bounded influence" I mean that the contribution to the estimate for $\sigma^2$ from the ith observation cannot exceed a given threshold (this is $(\nu+1)\sigma^2_{old}$ in the EM algorithm). Also $\nu$ is a "robustness"parameter in that increasing (decreasing) $\nu$ will result in more (less) uniform weights and hence more (less) sensitivity to outliers.

One thing to note is that the log likelihood function may have more than one stationary point, so the EM algorithm may converge to a local mode instead of a global mode. The local modes are likely to be found when the location parameter is started too close to an outlier. So starting at the median is a good way to avoid this.

• That's awesome. I've been toying with the idea of fitting student t's using EM for a while for precisely the reason that it looks like a mixture of gaussians. Do you have a citation/reference for the update equations you give? Having that would increase the awesomeness of this post even further.
– Pat
Jul 10, 2013 at 8:37
• Actually, i think I've found one myself, for a mixture model of student t's (which I'm so going to use for stuff): The mixtures of Student’s t-distributions as a robust framework for rigid registration. Demetrios Gerogiannis, Christophoros Nikou, Aristidis Likas. Image and Vision Computing 27 (2009) 1285–1294.
– Pat
Jul 10, 2013 at 8:46
• The link in my answer to this question has a very general EM framework for loads and loads of likelihood functions - quantile, student, logistic, and does general regression. You specific case is "regression" without covariates - intercept only - so fits nicely into this framework. Also, there are a vast number of penalty terms that you can incorporate into this framework. Jul 12, 2013 at 2:01
• @probabilityislogic really neat! And what if $\nu$ is also unknown? Can you also give some reference please? Maybe best here: stats.stackexchange.com/questions/87405/… Feb 21, 2014 at 12:17
• I think this reference is better than @Pat's. 'ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME.' You have to be very careful on the selection of initial parameter value while running the EM algorithm because of the local-optimum issue. In other words, you have to know something about your data. Usually, I avoid the use of the t distribution in my research.
– user46849
Jun 6, 2014 at 9:28

The following paper addresses exactly the problem you posted.

Liu C. and Rubin D.B. 1995. "ML estimation of the t distribution using EM and its extensions, ECM and ECME." Statistica Sinica 5:19–39.

It provides a general multivariate t-distribution parameter estimation, with or without the knowledge of the degree of freedom. The procedure can be found in Section 4, and it is very similar to probabilityislogic's for 1-dimension.

• It sounds like the paper you refer to contains a useful answer to the question, but answers are better when they are standalone and do not necessitate outside resources (here, for example, it is possible that OP or readers do not have access to this paper). Could you flesh out your answer a little bit to make it more standalone? Nov 25, 2015 at 23:47

I doubt that it exists in closed form: if you write any one of the factors of the likelihood as $$\frac{\Gamma(\frac{\nu+1}{2})} {\sqrt{\nu\pi}\,\Gamma(\frac{\nu}{2})} \left(1+\frac{t^2}{\nu} \right)^{-\frac{\nu+1}{2}} = \frac{\Gamma(\frac{\nu+1}{2})} {\sqrt{\nu\pi}\,\Gamma(\frac{\nu}{2})} \exp \left \{ \left [ \ln \left(1+\frac{t^2}{\nu} \right) \right ] \left [ {-\frac{\nu+1}{2}} \right ]\right \}$$ and take the ln of that, you will get a nonlinear equation in $\nu$. Even if you manage to get a solution, then depending on the number of factors (terms) $n$, the MLE equation is going to depend on this $n$ in a nontrivial way. All that dramatically simplifies, of course, when $\nu \rightarrow \infty$, when the power approaches an exponential (Gaussian PDF).

• Even in the Gaussian setting the log likelihood is nonlinear in its parameters :-).
– whuber
Jul 8, 2013 at 20:05
• I am actually interested in location and scale parameters, more than in the degrees of freedom. Please see edit to the question, and sorry for being not precise. Jul 9, 2013 at 9:54

Does a closed-form maximum-likelihood estimator for the Student's t distribution exist? The answer is now YES!! During the COVID pandemic, I dug into this problem and discovered a method I call Independent Approximators (IAs). This new algorithm provides a closed-form estimate of the location, scale, and shape that achieves the maximum likelihood estimate. The method works by filtering the samples by pairs and triplets that are approximately equal. The IA-pairs are distributed as the normalized square of the original distribution and are guaranteed to have a defined mean. The IA-triplets are distributed as the normalized cube of the original distribution, are guaranteed to have a finite second moment, and are used to estimate the scale. Finally, the geometric mean is used to estimate the scale, as defined in a paper I posted earlier to the question.

Try it out. I'm quite interested in getting feedback on this new method. Mathematica code is available in the referenced Github repository.

Kenric Nelson, "Independent Approximates enable closed-form estimation of heavy-tailed distributions"

https://arxiv.org/abs/2012.11026