4
$\begingroup$

I have a question regarding Maximum Likelihood Estimate in linear regression model without intercept. I have a model: $$Y_i=\beta X_i +\epsilon_i, \ \ i=1,...,n$$ where $\epsilon_i$ are i.i.d. $N(0, \sigma^2)$

I concluded that the log-likelihood function looks like this: $$l(\beta,\sigma)=\sum_{i=1}^n \left( \ln(\frac{1}{\sqrt{2\pi}}) -\ln(\sigma) -\frac{(y_i-(\beta x_i))^2}{2\sigma^2} \right)$$ Easy part of this question is MLE of $\sigma$ and MLE of $\beta$. But what I really don't know how to evaluate is MLE of $\frac{\beta}{\sigma}$. What crossed my mind is to only set MLE of $\frac{\beta}{\sigma}=\frac{\beta_{MLE}}{\sigma_{MLE}}$. But I believe that this is not the right solution.

Any hints?

$\endgroup$
1
  • $\begingroup$ I have provided a hint as an answer. Also, this looks like a homework problem, so if it is, please add the [self-study] tag to ensure appropriate discussion of the solution. $\endgroup$ Commented Apr 11, 2016 at 19:25

2 Answers 2

5
$\begingroup$

A maximum likelihood estimator has the nice property that it is invariant under transformations. This means that if $\theta_{MLE}$ is the MLE for $\theta$, then for a function $g(\theta)$, $g(\theta_{MLE})$ is the MLE for $g(\theta)$.

This can be directly applied to your problem. Hint: what is the MLE for $(\beta, \sigma)$?

$\endgroup$
2
  • 1
    $\begingroup$ Then, if I understand correctly, MLE of that ratio really is just the ratio of MLE estimates. Then my idea was not that far away from truth.. Thanks! And of course, next time I will include self-study tag :) $\endgroup$
    – Juraj
    Commented Apr 11, 2016 at 20:13
  • 2
    $\begingroup$ Juraj, it may help future visitors if you were to mark this answer as accepted: that influences site searches, for instance. You also get a little bump in reputation for the effort :-). $\endgroup$
    – whuber
    Commented Apr 13, 2016 at 20:56
1
$\begingroup$

Greenparker has already answered the substance of your question. I will just add that it is also a good idea to simplify your log-likelihood as much as possible before proceeding with maximisation. Removing the additive constant and simplifying the remaining sums gives:

$$l(\beta,\sigma)= -n \ln (\sigma) - \frac{1}{2 \sigma^2} \sum_{i=1}^n (y_i-\beta x_i)^2 = -n \ln (\sigma) - \frac{1}{2 \sigma^2} || \boldsymbol{y} - \beta \boldsymbol{x} ||^2 .$$

Conditional on $\hat{\sigma}$ you need only minimise the quantity $|| \boldsymbol{y} - \beta \boldsymbol{x} ||^2$. You are correct that once you have found the MLEs for the individual parameters, the MLE of their ratio is the ratio of the MLEs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.