1
$\begingroup$

In Likelihood and All That Ben Bolker states " the joint likelihood of the whole data set is the product of the likelihoods of each individual observation". Is there a way that someone could explain to me, in simple terms, how we obtain the likelihood of an individual observation?

$\endgroup$
2
$\begingroup$

The likelihood is what you get when you evaluate the appropriate probability density function (using the model and parameter values of interest) at the value of the observation.

$\endgroup$
7
  • $\begingroup$ Thank you @ Kodiologist. Sounds so simple when you say it like that. So in NHST is the shape and central tendency of that density function (which determines the likelihood of each observation) also determined by the data? So a normal distribution centred on a the null (e.g. mean difference of 0, if mean difference is the parameter we are interested in) with a standard deviation that is the standard error of the sample? $\endgroup$
    – llewmills
    Aug 1 '16 at 11:58
  • 1
    $\begingroup$ Most significance tests don't involve likelihood computation, the likelihood-ratio test being an exception. The commonest setting where likelihood appears is maximum likelihood estimation. $\endgroup$ Aug 1 '16 at 14:23
  • $\begingroup$ So when I get a parameter estimate such as a slope or intercept in a program like nlme is that derived using Maximum Likelihood Estimation? i ask because I also get a p-value. $\endgroup$
    – llewmills
    Aug 2 '16 at 0:56
  • 1
    $\begingroup$ It depends. Maximum likelihood estimation is a popular way to estimate parameters, and perhaps the single most important one, but it isn't the only one. For example, glns in R's nlme package uses generalized least squares, which doesn't involve MLE, I think. $\endgroup$ Aug 2 '16 at 1:50
  • 1
    $\begingroup$ That is true for a discrete random variable, but not true in general. For example, under a standard normal distribution, the likelihood of 0 is $1/\sqrt{2\pi} ≈ .4$, whereas its probability is 0, as for any single value of a continuous distribution. $\endgroup$ Aug 2 '16 at 3:55
2
$\begingroup$

Let's play a game that you might solve by maximum likelihood:

Assume that a someone is in a room, you can not see the person. The person has a die and a coin and decides to try one of these.

If he trows the die and observes a '1' then he says you that he has '1' else he says you he has zero. If he throws the coin then he says that he has '1' when it ends head up and '0' otherwise.

He does not tell you whether he threw the coin or the die, you can not see it neither. The only thing that he tells you is whether he has one or zero.

Assume that he says it is 1. What would you guess, did he throw the die or the coin ?

What is the likelihood of this observation ?

  • if the coin has been thrown, then the probability that you have a one, given that is was the coin that was tossed, is equal to 0.5 or 50%, or the likelihood of having a coin, given that he says it is '1', is 50% (note that this is a value of the parameter of the coin)
  • If the die has been thrown, then the probability that he says '1', given that the die was thrown is 1/6=16.666%. Or the likelihood that it was a die given that he says '1' is 16.666% (note that this is a value of the parameter of the die).

If you have to take a decision based on the maximum likelihood principle then you decide for the one with the highest likelihood, or you decide that he has thrown a coin (because the likelihood of the coin given that he says '1' is 50%, which is higher than the likelihood that he threw a die given that he says 1 (16.6666%)).

$\endgroup$
1
$\begingroup$

Suppose there are three observations: 3, 4, 8 (and they are independent). In this example, each of the three numbers is an individual observation. The whole data set is represented by the three observations.

Because, I think, a specific example makes it easier to understand... If we assume that the data (3, 4, and 8) came from a Poisson distribution with mean 5. The likelihoods of the observations (3, 4, and 8), respectively, are dpois(3,lambda=5), dpois(4,lambda=5), dpois(8,lambda=5).

The joint likelihood of the whole data set is

dpois(3,lambda=5)*dpois(4,lambda=5)*dpois(8,lambda=5)

The example is consistent with the statement. dpois(3,lambda=5), dpois(4,lambda=5), and dpois(8,lambda=5) are the likelihood of an individual observation.

$\endgroup$
2
  • $\begingroup$ Thank you @quibble. That was a nice neat example. So am I right in guessing that you didn't select the assumed mean of 5 for the distribution by accident? i.e. that the mean of the poisson distribution from which you obtained your likelihood estimates for each observation was obtained from the mean of the sample? $\endgroup$
    – llewmills
    Aug 2 '16 at 5:43
  • 1
    $\begingroup$ In this case, lambda=5 is the MLE ((3+4+8)/3=5), but the example works for any feasible value of lambda (>0). For example, if you set lambda=7.0123 in the example, everything works. I picked the MLE by chance... In practice, you do not know lambda and tries to find the lambda that maximizes the joint likelihood of the whole data set. $\endgroup$
    – quibble
    Aug 2 '16 at 10:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.