We have the following simple linear model without the predictor variable: $y_t = \beta_t + \epsilon_t \quad \text{ where } t = 1, 2, \ldots,n; \ \epsilon_t \sim N(0, \sigma^2)$

What is the maximum likelihood estimator for $\beta_t$?

Proceeding with the log-likelihood function and removing the constant term and the term with $\sigma^2$, we find that the problem reduces to the following minimization problem:

$\min_{\beta_t}\sum_{t=1}^{n}(y_t - \beta_t)^2$

The maximum likelihood estimator for $\beta$ is therefore the corresponding $y$. Is the interpretation correct?

  • $\begingroup$ Is $\sigma^2$ known? $\endgroup$ – StubbornAtom Apr 10 '20 at 14:42
  • $\begingroup$ I think I have followed, and described, an MLE method, but agree that the final result for a single $\beta_{MLE}$ in our model, if single $\beta$ were the case, would be the same as the least square solution of single $\beta$. But it is the MLE interpretation about different $\beta$s that I am not fully certain yet about. $\endgroup$ – Mathophile Apr 10 '20 at 14:46
  • $\begingroup$ Yes, $\sigma^2$ is know. Say, it equals 1. $\endgroup$ – Mathophile Apr 10 '20 at 14:48

If you're estimating a separate $\beta_t$ parameter for each data point, the maximum likelihood solution is to set $\epsilon = 0$, and so $\sigma = 0$. This model can perfectly fit your data, but is useless, since it has one parameter per data point.

A more useful model is $y_t = \beta + \epsilon_t$, with only one $\beta$ parameter. The MLE of $\beta$ is just the mean of the $y$ values. The MLE of $\sigma^2$ isn't exactly the same as the sample variance, but it's close.

  • $\begingroup$ Thanks. I agree it's more interesting to have only one $\beta$ in the model for practical purpose. However, on a pure theoretical pursuit, if we have $\sigma \ne 0$ and hence $\epsilon = 0$ does not hold true, what can we say about the MLE of $\beta$? $\endgroup$ – Mathophile Apr 10 '20 at 14:18
  • $\begingroup$ $\sigma = 0, \beta_t = y_t$ is the MLE, and has a likelihood of 1, since all values of $y$ are exactly where the model predicts them to be. If you restrict your analysis to, for instance, $\sigma \geq .1$, then $\sigma = .1, \beta_t = y_t$ is the MLE. $\endgroup$ – Eoin Apr 14 '20 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.