9
$\begingroup$

Lifetime of 3 electronic components are $X_{1} = 3, X_{2} = 1.5,$ and $X_{3} = 2.1$. THe random variables had been modeled as a random sample of size 3 from the exponential distribution with parameter $\theta$. The likelihood function is, for $\theta > 0$

$f_{3}(x|\theta) = \theta^{3} exp(-6.6\theta)$, where $x = (2, 1.5, 2.1)$.

And then the problem proceeds to determine the M.L.E by finding the value of $\theta$ that maximizes $log f_{3}(x|\theta)$. My question is, how do I determine the likelihood function? I looked up the pdf of the exponential distribution, but it's different. So is the likelihood function always given to me in a problem? Or do I have to determine it myself? If so, how?

$\endgroup$
  • $\begingroup$ Why do you want to do likelihood estimation with only 3 observations? The estimate you get for $\theta$ will be biased and have huge amount of variance. Is it HW? $\endgroup$ – Zachary Blumenfeld Jan 10 '15 at 3:20
  • $\begingroup$ Do you know what the definition of the likelihood is? $\endgroup$ – Glen_b Jan 10 '15 at 3:32
14
$\begingroup$

The likelihood function of a sample, is the joint density of the random variables involved but viewed as a function of the unknown parameters given a specific sample of realizations from these random variables.

In your case, it appears that the assumption here is that the lifetime of these electronic components each follows (i.e. its has a marginal distribution), an exponential distribution with identical rate parameter $\theta$, and so the marginal PDF is:

$$f_{X_i}(x_i\mid \theta) = \theta e^{-\theta x_i}, \;\; i=1,2,3$$

Also, it appears that the life of each component is fully independent of the life of the others. In such a case the joint density function is the product of the three densities,

$$f_{X1,X2,X3}(x_1,x_2,x_3\mid \theta) = \theta e^{-\theta x_1} \cdot \theta e^{-\theta x_2}\cdot \theta e^{-\theta x_3} = \theta^3\cdot \exp{\left\{-\theta \sum_{i=1}^3x_i\right\}}$$

To turn this into the likelihood function of the sample, we view it as a function of $\theta$ given a specific sample of $x_i$'s.

$$ L(\theta \mid \{x_1,x_2,x_3\}) = \theta^3\cdot \exp{\left\{-\theta \sum_{i=1}^3x_i\right\}}$$

where only the left-hand-side has changed, to indicate what is considered as the variable of the function. In your case the available sample is the three observed lifetimes $\{x_1 = 3, x_2 =1.5, x_3 = 2.1\}$, and so $\sum_{i=1}^3x_i = 6.6$. Then the likelihood is

$$L(\theta \mid \{x_1 = 3, x_2 =1.5, x_3 = 2.1\}) = \theta^3\cdot \exp{\left\{-6.6\theta \right\}}$$

In other words, in the likelihood you were given, the specific sample available has been already inserted in it. This is not usually done, i.e. we usually "stop" at the theoretical representation of the likelihood for general $x_i$'s, we then derive the conditions for its maximization with respect to $\theta$, and then we plug into the maximization conditions the specific numerical sample of $x$-values, in order to obtain a specific estimate for $\theta$.

Admittedly though, looking at the likelihood like this, may make more clear the fact that what matters here for inference (for the specific distributional assumption), is the sum of the realizations, and not their individual values: the above likelihood is not "sample-specific" but rather "sum-of-realizations-specific": if we are given any other $n=3$ sample for which the sum of its elements is again $6.6$, we will obtain the same estimate for $\theta$ (this is essentially what it means to say that $\sum x$ is a "sufficient" statistic -it contains all information that the sample can provide for inference, under the specific distributional assumption).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.