Conditional vs. Unconditional Maximum Likelihood

Question

I have some questions on the difference between conditional MLE (CMLE) and unconditional MLE (UMLE) in practice. In what follows I will only talk about the unconditional and conditional mean and leave out the variance in order to shorten the question.

Example 1:

We have a linear model: $y_{i}=x_{i}^{\prime}\beta+\varepsilon_{i}$, $\varepsilon_{i}\sim N\left(0,\:\sigma^{2}\right)$

$y$ has conditional mean: $E\left[y\mid x\right]=x_{i}^{\prime}\beta$, and unconditional mean: $E\left[y\right]=\mu$

Am I right to assume that unconditional Maximum Likelihood estimation proceeds like this?

The density function for observation $y$ is: $f\left(y_{i}\right)=\left(\frac{1}{\sqrt{2\pi\sigma^{2}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{i}-\mu\right)^{2}}{\sigma^{2}}\right)$

The log-likelihood function is:$\log L\left(\mu,\:\sigma^{2}\right)=-\frac{N}{2}\log\left(2\pi\sigma^{2}\right)-\frac{1}{2}\sum_{i=1}^{N}\frac{\left(y_{i}-\mu\right)^{2}}{\sigma^{2}}$

And the UMLE estimator is: $\hat{\mu}_{ML}=\frac{1}{N}\sum_{i=1}^{N}y_{i}$

Am I right to assume that conditional Maximum Likelihood estimation proceeds like this? $f\left(y_{i}\mid x_{i}\right)=\left(\frac{1}{\sqrt{2\pi\sigma^{2}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{i}-x_{i}^{\prime}\beta\right)^{2}}{\sigma^{2}}\right)$

the log-likelihood function is: $\log L\left(\beta,\:\sigma^{2}\right)=-\frac{N}{2}\log\left(2\pi\sigma^{2}\right)-\frac{1}{2}\sum_{i=1}^{N}\frac{\left(y_{i}-x_{i}^{\prime}\beta\right)^{2}}{\sigma^{2}}$

The CMLE estimator is: $\hat{\beta}_{ML}=\frac{\sum_{i=1}^{N}x_{i}y_{i}}{\sum_{i=1}^{N}x_{i}^{2}}$

Example 2

We have a Poisson model:

$y$ has conditional mean $E\left[y_{i}\mid x_{i}\right]=\lambda_{i}=\exp\left(x_{i}^{\prime}\beta\right)$ and unconditional mean: $E\left[y\right]=\lambda$.

Am I right to assume that unconditional Maximum Likelihood estimation proceeds like this?

The density function for $y$ is: $f\left(y_{i}\right)=\frac{e^{-\lambda}\lambda^{y_{i}}}{y_{i}!}$

The log-likelihood function for is: $\log L\left(\lambda\right)=\sum_{i=1}^{N}log\left(\frac{e^{-\lambda}\lambda^{y_{i}}}{y_{i}!}\right)=\sum_{i=1}^{N}\left(-\lambda+y_{i}\log\left(\lambda\right)-\log\left(y_{i}\right)\right)$

And the UMLE estimator is: $\hat{\lambda}_{ML}=\frac{1}{N}\sum_{i=1}^{N}y_{i}$

Am I right to assume that conditional Maximum Likelihood estimation proceeds like this? $f\left(y_{i}\mid x_{i}\right)=\frac{e^{-\lambda}\lambda^{y_{i}}}{y_{i}!}=\frac{\exp\left(-\exp\left(x_{i}^{\prime}\beta\right)\right)\cdot\left[\exp\left(x_{i}^{\prime}\beta\right)\right]^{y_{i}}}{y_{i}!}$

The log-likelihood function is: $\log L\left(\lambda\right)=\sum_{i=1}^{N}\left(\frac{\exp\left(-\exp\left(x_{i}^{\prime}\beta\right)\right)\cdot\left[\exp\left(x_{i}^{\prime}\beta\right)\right]^{y_{i}}}{y_{i}!}\right)=\sum_{i=1}^{N}\left(-\exp\left(x_{i}^{\prime}\beta\right)+y_{i}x_{i}^{\prime}\beta-log\left(y_{i}!\right)\right) $

And the CMLE estimator is the solution to: $\sum_{i=1}^{N}\left(y_{i}-\exp\left(x_{i}^{\prime}\beta\right)\right)x_{i}^{\prime}=0$

Example 3

We have an AR(1) model: $y_{t}=\mu+\Theta y_{t}+\varepsilon_{t}$, $\varepsilon_{t}\sim N\left(0,\:\sigma^{2}\right)$

$y$ has conditional mean: $E\left[y\mid y_{t-1}\right]=\mu+\Theta y_{t}$, and unconditional mean: $E\left[y\right]=\frac{\mu}{1-\Theta}$

$y$ has conditional variance: $Var\left[y_{t}\mid y_{t-1}\right]=\sigma^{2}$ and unconditional variance: $Var\left[y_{t}\right]=\frac{\sigma^{2}}{1-\Theta^{2}}$

Am I right to assume that unconditional Maximum Likelihood estimation proceeds like this?

The unconditional density function for $y$ is: $f\left(y_{t}\right)=\left(\frac{1}{\sqrt{2\pi\frac{\sigma^{2}}{1-\Theta^{2}}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{t}-\frac{\mu}{1-\Theta}\right)^{2}}{\frac{\sigma^{2}}{1-\Theta^{2}}}\right)$

and then I state the log-likelihood function and derive the estimators as above.

Am I right to assume that conditional Maximum Likelihood estimation proceeds like this?

The conditional density function for $y$ is: $f\left(y_{t}\mid y_{t-1}\right)=\left(\frac{1}{\sqrt{2\pi\sigma^{2}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{i}-\mu-\Theta y_{t-1}\right)^{2}}{\sigma^{2}}\right)$

and then I proceed by stating the log-likelihood function and deriving the estimators as above.

My question is if what I have done above is correct? I know that the CMLE is right since I condition on x. The question is if the UMLE part is correct? I am quite sure that Example 3 is correct but I am not so sure Example 1 and Example 2 are correct.

Any help would be greatly appreciated.

Answer 1

The main reason for using conditional maximum likelihood is the resulting distribution. For Y|X ~N(x'B,Var(eps)) holds because the variation of Y only depends on the (normal) variation of the eps. As you know the correct assumption of the underlying density is a crucial point in MML estimation and hence, with unconditional MML you would run into problems here. Think about how the unconditional distribution of Y would look like if X is for instance N(3, Var(eps)). Y would have a mixture distribution that is not even normal anymore. The density function that you write for unconditional likelihood (example 1) is therefore wrong and moreover, the variance of Y is definitely not σ2. If you don't condition on X the variation of X needs to be accounted for in the variation of Y. Again, the variation of X needs to be considered if you are using unconditional MML.