3
$\begingroup$

I was going through a lecture slides on maximum likelihood estimation (MLE) and bumped into something I couldn't understand. The lecturer was using MLE to estimate a function $$ y_i=Q(\alpha , \beta ,x_i). $$ Now during MLE estimation (estimating parameters $\alpha$ and $\beta$), the term $$ y_i-Q(\alpha ,\beta,x_i)\quad (1) $$ is directly plugged into Gaussian distribution equations as a random variable. $$ L=\prod\limits_{i=1}^n {\frac{1}{\sigma_i \sqrt{2 \pi}}} \exp{\left(-\frac{{(y_i-Q(\alpha,\beta,x_i))}^2}{2 \sigma_i^2}\right)} $$ Shouldn't $(1)$ be a result of the distribution with random variable as $x$, assuming that the variable $y$ has some noise present in it ?

$\endgroup$
1
  • 6
    $\begingroup$ Most certainly, $y_i=Q(\alpha , \beta ,x_i) +u_i \Rightarrow y_i-Q(\alpha , \beta ,x_i) =u_i$ and not just "some noise", but noise for which we assume that it follows the normal distribution and that it is independently distributed (otherwise the likelihood function wouldn't be just the product of the individual densities). $\endgroup$ Sep 29 '13 at 10:59
3
$\begingroup$

To close this one:

Lecture slides usually are not drafted in order to tell the whole story on their own, and so in many cases require verbal input from the lecturer to provide a complete picture. In this specific case, probably the lecturer would:

1) Inform the audience that $y_i=Q(\alpha , \beta ,x_i)$ is the "deterministic" version of a relation that comes from theoretical concerns that usually assume away various irregularities that are present in the real world, and so, that for estimation purposes the equation will be supplied by an "error term"

2) Consequently, inform the audience that an assumption is made that this error term is a random variable following the normal distribution, with mean $0$ and variance $\sigma^2_i$ (heteroskedastic and so non-identically distributed), and also, that it is independently distributed across $i$... which would permit him to show the likelihood function presented at the end of the question here.

$\endgroup$
3
  • $\begingroup$ Great answer, thanks :) (upvoted in the past). I come up with a doubt: Under these assumptions of Gaussian noise in the errors and independence between them, could it be said that the estimated parameters $\alpha$ and $\beta$ are Gaussian distributed? $\endgroup$
    – Javier TG
    Jan 15 at 18:21
  • 1
    $\begingroup$ @JavierTG The formulation seems to imply that the $Q$ function is non-linear. In such a situation I believe Normality of the estimator is established only asymptotically, in which case the distribution of the error term does not matter. If the Q function was linear, then Gaussian MLE would equal OLS with a Gaussian error term and then Normality of the estimator would hold also in finite samples. $\endgroup$ Jan 16 at 19:14
  • $\begingroup$ Thanks a lot for the insightful answer! $\endgroup$
    – Javier TG
    Jan 16 at 19:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.