How is to maximize a function $f(x,y)$ for values of $y$?

Question

Maximum Likelihood Method:

Likelihood function asks what value of parameter, $\theta$, makes the data set most probable.

Let the distribution is

$$f(x;p)=\binom{3}{x}p^x(1-p)^{n-x},\quad x=0,1,2,3.$$

Again, suppose the parameter $p$ can assume one of the two values $\frac{3}{4}$ or $\frac{1}{4}$.

The possible outcomes and their probabilities are given below:

$$ \begin{array}{c|ccc} x & 0 & 1 & 2 & 3 \\ \hline f(x;\frac{3}{4}) & \frac{1}{64} & \frac{9}{64} & \frac{27}{64} & \frac{27}{64} \\ f(x;\frac{1}{4}) & \frac{27}{64} & \frac{27}{64} & \frac{9}{64} & \frac{1}{64} \end{array} $$

The maximum likelihood estimator may be defined as

$$\hat p=\hat p(x) = \begin{cases} .25, & \text{for $x=0,1$} \\[2ex] .75, & \text{for $x=2,3$}. \end{cases}$$

Profile Likelihood Method:

Suppose we have two parameters, $\theta$ and $\delta$, where $\theta$ is of interest and $\delta$ is a nuisance parameter.

The profile likelihood of $\theta$ is

$$L_p(\theta)=\max_\delta L(\theta,\delta),$$

where $L(\theta,\delta)$ is the "complete likelihood".

$\max_\delta L(\theta,\delta)$ expresses maximize the likelihood function of $\theta$ and $\delta$, $L(\theta,\delta)$, over $\delta$ values.

My question is:

How is to maximize a function over values of a variable? That is, how is to maximize a function $f(x,y)$ for values of $y$?

Answer 1

Continuous parameters can be handled with differentiation and analysis. If the parameters are discrete as in your first example, you first fix $\theta$ and $x$, then for all $\delta$, you'll search for maximum values of the likelihood, and finally you'll obtain a discrete function $L_p(\theta,x)$ as a table.