0
$\begingroup$

We have the following simple linear model without the predictor variable: $y_t = \beta_t + \epsilon_t \quad \text{ where } t = 1, 2, \ldots,n; \ \epsilon_t \sim N(0, \sigma^2)$

What is the maximum likelihood estimator for $\beta_t$?

Proceeding with the log-likelihood function and removing the constant term and the term with $\sigma^2$, we find that the problem reduces to the following minimization problem:

$\min_{\beta_t}\sum_{t=1}^{n}(y_t - \beta_t)^2$

The maximum likelihood estimator for $\beta$ is therefore the corresponding $y$. Is the interpretation correct?

$\endgroup$
3
  • $\begingroup$ Is $\sigma^2$ known? $\endgroup$ Commented Apr 10, 2020 at 14:42
  • $\begingroup$ I think I have followed, and described, an MLE method, but agree that the final result for a single $\beta_{MLE}$ in our model, if single $\beta$ were the case, would be the same as the least square solution of single $\beta$. But it is the MLE interpretation about different $\beta$s that I am not fully certain yet about. $\endgroup$
    – Mathophile
    Commented Apr 10, 2020 at 14:46
  • $\begingroup$ Yes, $\sigma^2$ is know. Say, it equals 1. $\endgroup$
    – Mathophile
    Commented Apr 10, 2020 at 14:48

1 Answer 1

3
$\begingroup$

If you're estimating a separate $\beta_t$ parameter for each data point, the maximum likelihood solution is to set $\epsilon = 0$, and so $\sigma = 0$. This model can perfectly fit your data, but is useless, since it has one parameter per data point.

A more useful model is $y_t = \beta + \epsilon_t$, with only one $\beta$ parameter. The MLE of $\beta$ is just the mean of the $y$ values. The MLE of $\sigma^2$ isn't exactly the same as the sample variance, but it's close.

$\endgroup$
2
  • $\begingroup$ Thanks. I agree it's more interesting to have only one $\beta$ in the model for practical purpose. However, on a pure theoretical pursuit, if we have $\sigma \ne 0$ and hence $\epsilon = 0$ does not hold true, what can we say about the MLE of $\beta$? $\endgroup$
    – Mathophile
    Commented Apr 10, 2020 at 14:18
  • $\begingroup$ $\sigma = 0, \beta_t = y_t$ is the MLE, and has a likelihood of 1, since all values of $y$ are exactly where the model predicts them to be. If you restrict your analysis to, for instance, $\sigma \geq .1$, then $\sigma = .1, \beta_t = y_t$ is the MLE. $\endgroup$
    – Eoin
    Commented Apr 14, 2020 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.