3
$\begingroup$

Suppose you have $\theta=\{1,2\}$ and the sample of (0,1,2) with the task of finding MLE:

\begin{array} {|c|c|c|} \hline x & p(x|\theta=1) & p(x|\theta=2) \\ \hline 0 & 1/2 & 1/4 \\ \hline 1 & 0 & 1/4 \\ \hline 2 & 1/3 & 0 \\ \hline \end{array}

I can't compare $\prod_i p(x_i|\theta)$, because of the zeroes. Even though I can compare probability for a single observation, I'm still having trouble quantifying the difference. $p(x=1|\theta=2)>p(x=1|\theta=1)$ for x=1, but can I do a similar comparison for the whole vector?

Edit

The original question is

One observation is taken on a discrete random variable X with pmf f(x|θ), where θ∈{1,2,3}. Find the MLE of θ.

This is followed by a 4x5 table which includes zeroes in every column. I think the idea behind this question is that we have a distribution that doesn't describe population well. Do we just say that the method of MLE is not applicable here?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ You should include the sample in your question. As whuber said, the likelihood is trivially zero for both parameter values ... which simply means neither parameter value can have produced the values $(0,1,2)$ (But neither conditional distribution seems to sum to 1 either. Is something missing?) Or are you supposed to treat each value as a single sample (giving three MLEs for three different samples)?? $\endgroup$
    – Glen_b
    Mar 17, 2014 at 20:31
  • $\begingroup$ I have changed the terminology. This question is similar to the one in my textbook. There is a sample of (0,1,2), in which case the conditional probabilities don't have to sum up to 1. $\endgroup$
    – m33lky
    Mar 17, 2014 at 20:37
  • $\begingroup$ Notice that the quotation begins "one observation." Thus, it's really a set of questions: what is the MLE when you observe $0$? What is it when you observe $1$? What is it when you observe $2$? $\endgroup$
    – whuber ♦
    Mar 17, 2014 at 21:43
  • $\begingroup$ @Glen_b put another way, each parameterization is equally likely e.g. impossible given the sample of 0, 1, 2. So both $\theta=1$ and $\theta=2$ are the non-unique MLEs. $\endgroup$
    – AdamO
    Mar 21, 2014 at 18:25

2 Answers 2

Reset to default
3
+50
$\begingroup$

Now we have the original question, the mystery is no mystery.

In that question, a single observation will be consistent (to greater or lesser degree) with some of the $\theta$ values and inconsistent with others (the ones that have no chance of giving that observed value).

So for example, if we look at your probability distributions, and possible samples of a single observation:

$x=0:\, $ the ML estimate is $\theta=1$ (since $\frac{1}{2}>\frac{1}{4}$)

$x=1:\, $ the ML estimate is $\theta=2$

$x=2:\, $ the ML estimate is $\theta=1$

So whichever outcome is observed the ML estimate is obvious. This (possibly in table form) is what the original question is looking for.

If instead of one observation, you have a sample of several (as in your version of the question), then a value of $\theta$ will be "ruled out" (have likelihood 0) by any observation in the sample having 0 probability under that $\theta$. If, as above, at least one sample value has likelihood zero under every value of $\theta$, all $\theta$ values have likelihood 0, as was previously suggested in comments.

Improve this answer
$\endgroup$
8
  • $\begingroup$ I'm using my formulation of the question. So, are all $\theta$ values ruled out or can we choose any one of them, since they are equally likely? $\endgroup$
    – m33lky
    Mar 22, 2014 at 2:44
  • 1
    $\begingroup$ I'm at a loss to understand how I failed to be clear. None of the $\theta$ values listed can produce a sample {1, 2, 3}. Forget statistics and imagine instead this experiment: I have an ordinary 6-sided die, and a coin with sides labelled "1" and "2". One of the two devices is to be selected and tossed. You will work out the chances for me after I observe the result, but you have to be elsewhere right away, so I will send you the result (just before I leave on vacation). I write outcome on a postcard and send it to you. When you get the card, it says "8". Which is more likely to give 8? $\endgroup$
    – Glen_b
    Mar 22, 2014 at 4:47
  • $\begingroup$ You saw the die and the coin, they were definitely marked correctly. If I asked you to bet (at good odds, I'll give you 3-1 if you guess correctly) on one of them being the one that generated the "8", would you say "oh, they're equally likely, it doesn't matter which I choose"? Or would you say something more like "What? No that simply can't happen! Something weird is going on! The situation cannot be as it was described. Was there another die or something?" $\endgroup$
    – Glen_b
    Mar 22, 2014 at 4:52
  • $\begingroup$ In my post I talk about how the distribution may not describe the population well. Consider the setting where the die is 8-sided, but we have pmf for a regular die. Using common sense you would choose the die, even though under your model none of the devices could have generated 8. $\endgroup$
    – m33lky
    Mar 22, 2014 at 17:48
  • $\begingroup$ Either I misunderstand the circumstances you're describing or your final comment makes no sense to me. How does the observation of '8' make a six-sided die a better choice than the coin? Neither can possibly give an 8. Maybe it was instead a draw from a deck of cards. Maybe using both and summing the result was an (unmentioned at the time) possibility. Maybe it was meant to be a "2" but somehow ended up looking like an 8 on the postcard. Maybe there was some other mistake. All we know is that if it is 8, none of the offered options is at all plausible. $\endgroup$
    – Glen_b
    Mar 22, 2014 at 22:41
1
$\begingroup$

If $P(x=1|\theta = 1) = 0$, then (0,1,2) is impossible, since it has a 1 in it. If $P(x=2|\theta = 2) = 0$, then (0,1,2) is impossible, since it has a 2 in it. So, your likelihood calculation is telling it like it is. It is impossible to get that sample, given your probability distributions and thetas.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.