Maximum Likelihood Method:

Likelihood function asks what value of parameter, $\theta$, makes the data set most probable.

Let the distribution is

$$f(x;p)=\binom{3}{x}p^x(1-p)^{n-x},\quad x=0,1,2,3.$$

Again, suppose the parameter $p$ can assume one of the two values $\frac{3}{4}$ or $\frac{1}{4}$.

The possible outcomes and their probabilities are given below:

$$ \begin{array}{c|ccc} x & 0 & 1 & 2 & 3 \\ \hline f(x;\frac{3}{4}) & \frac{1}{64} & \frac{9}{64} & \frac{27}{64} & \frac{27}{64} \\ f(x;\frac{1}{4}) & \frac{27}{64} & \frac{27}{64} & \frac{9}{64} & \frac{1}{64} \end{array} $$

The maximum likelihood estimator may be defined as

$$\hat p=\hat p(x) = \begin{cases} .25, & \text{for $x=0,1$} \\[2ex] .75, & \text{for $x=2,3$}. \end{cases}$$

Profile Likelihood Method:

Suppose we have two parameters, $\theta$ and $\delta$, where $\theta$ is of interest and $\delta$ is a nuisance parameter.

The profile likelihood of $\theta$ is

$$L_p(\theta)=\max_\delta L(\theta,\delta),$$

where $L(\theta,\delta)$ is the "complete likelihood".

$\max_\delta L(\theta,\delta)$ expresses maximize the likelihood function of $\theta$ and $\delta$, $L(\theta,\delta)$, over $\delta$ values.

My question is:

How is to maximize a function over values of a variable? That is, how is to maximize a function $f(x,y)$ for values of $y$?


Continuous parameters can be handled with differentiation and analysis. If the parameters are discrete as in your first example, you first fix $\theta$ and $x$, then for all $\delta$, you'll search for maximum values of the likelihood, and finally you'll obtain a discrete function $L_p(\theta,x)$ as a table.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.