0
$\begingroup$

Say if we wanted to conduct MLE to obtain the parameter estimate for a simple linear regression model

e.g.

wage = β1educ

I understand that we first find the joint probability of y taking on certain values (given certain x values).

E.g. if we had a sample of data with the following results: wage=10 and educ=2, wage=15 and educ=4, wage=20 and educ=6, etc, etc. The first step of MLE would be to compute the joint probability of all these sample values occurring.

If we have assumed that the errors are normally distributed then I have been taught that we do:

enter image description here

Notably, the equation inside the product operator is the formula for the PDF.

Finally, my question is: If we can't use a PDF to say that the probability of y is exactly equal to something, then why are we doing that here?

In my example to compute the MLE we would be essentially saying that the P(y=10|x=2) is equal to some exact value given by the PDF, P(y=15|x=4) is equal to some exact value given by the PDF, etc, etc... and then obviously we multiply these probabilities together to get the likelihood. So my confusion is around the fact that we are saying that an exact probability of something occurring is given by a value on a PDF when I thought that was impossible.

$\endgroup$
  • $\begingroup$ Is your confusion just that you're incorrectly interpreting the probability density as a probabiility? $\endgroup$ – AdamO Sep 14 at 13:37
  • $\begingroup$ @William: Hi: That's a great question. I don't know the exact answer but the idea is that each event you describe belongs to a set which has zero measure with respect to the lebesgue integral. But the likelihood that that event can happen is not-zero. Hopefully someone else can explain in more detail because I'd be interested in understanding it more clearly myself. $\endgroup$ – mlofton Sep 14 at 13:37
  • $\begingroup$ @Adam0 what is the difference between the two? I’m still trying to get the hang of probability in general. $\endgroup$ – William Sep 15 at 4:22
0
$\begingroup$

The likelihood doesn't deal with probabilities of exact values unless the sample space of the "Y" is discrete. A probability density, like the normal PDF, doesn't give us probabilities of exact values, but I know that a density of 0.25 is more "likely" than a density of 0.1. That is the quantity we maximize.

$\endgroup$
  • $\begingroup$ Ahhh, right. So the values on the y axis of the PDF are "likelihoods", rather than actual exact probabilities. $\endgroup$ – William Sep 16 at 7:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.