3
$\begingroup$

I have some questions on the difference between conditional MLE (CMLE) and unconditional MLE (UMLE) in practice. In what follows I will only talk about the unconditional and conditional mean and leave out the variance in order to shorten the question.

Example 1:

We have a linear model: $y_{i}=x_{i}^{\prime}\beta+\varepsilon_{i}$, $\varepsilon_{i}\sim N\left(0,\:\sigma^{2}\right)$

$y$ has conditional mean: $E\left[y\mid x\right]=x_{i}^{\prime}\beta$, and unconditional mean: $E\left[y\right]=\mu$

Am I right to assume that unconditional Maximum Likelihood estimation proceeds like this?

The density function for observation $y$ is: $f\left(y_{i}\right)=\left(\frac{1}{\sqrt{2\pi\sigma^{2}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{i}-\mu\right)^{2}}{\sigma^{2}}\right)$

The log-likelihood function is:$\log L\left(\mu,\:\sigma^{2}\right)=-\frac{N}{2}\log\left(2\pi\sigma^{2}\right)-\frac{1}{2}\sum_{i=1}^{N}\frac{\left(y_{i}-\mu\right)^{2}}{\sigma^{2}}$

And the UMLE estimator is: $\hat{\mu}_{ML}=\frac{1}{N}\sum_{i=1}^{N}y_{i}$

Am I right to assume that conditional Maximum Likelihood estimation proceeds like this? $f\left(y_{i}\mid x_{i}\right)=\left(\frac{1}{\sqrt{2\pi\sigma^{2}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{i}-x_{i}^{\prime}\beta\right)^{2}}{\sigma^{2}}\right)$

the log-likelihood function is: $\log L\left(\beta,\:\sigma^{2}\right)=-\frac{N}{2}\log\left(2\pi\sigma^{2}\right)-\frac{1}{2}\sum_{i=1}^{N}\frac{\left(y_{i}-x_{i}^{\prime}\beta\right)^{2}}{\sigma^{2}}$

The CMLE estimator is: $\hat{\beta}_{ML}=\frac{\sum_{i=1}^{N}x_{i}y_{i}}{\sum_{i=1}^{N}x_{i}^{2}}$

Example 2

We have a Poisson model:

$y$ has conditional mean $E\left[y_{i}\mid x_{i}\right]=\lambda_{i}=\exp\left(x_{i}^{\prime}\beta\right)$ and unconditional mean: $E\left[y\right]=\lambda$.

Am I right to assume that unconditional Maximum Likelihood estimation proceeds like this?

The density function for $y$ is: $f\left(y_{i}\right)=\frac{e^{-\lambda}\lambda^{y_{i}}}{y_{i}!}$

The log-likelihood function for is: $\log L\left(\lambda\right)=\sum_{i=1}^{N}log\left(\frac{e^{-\lambda}\lambda^{y_{i}}}{y_{i}!}\right)=\sum_{i=1}^{N}\left(-\lambda+y_{i}\log\left(\lambda\right)-\log\left(y_{i}\right)\right)$

And the UMLE estimator is: $\hat{\lambda}_{ML}=\frac{1}{N}\sum_{i=1}^{N}y_{i}$

Am I right to assume that conditional Maximum Likelihood estimation proceeds like this? $f\left(y_{i}\mid x_{i}\right)=\frac{e^{-\lambda}\lambda^{y_{i}}}{y_{i}!}=\frac{\exp\left(-\exp\left(x_{i}^{\prime}\beta\right)\right)\cdot\left[\exp\left(x_{i}^{\prime}\beta\right)\right]^{y_{i}}}{y_{i}!}$

The log-likelihood function is: $\log L\left(\lambda\right)=\sum_{i=1}^{N}\left(\frac{\exp\left(-\exp\left(x_{i}^{\prime}\beta\right)\right)\cdot\left[\exp\left(x_{i}^{\prime}\beta\right)\right]^{y_{i}}}{y_{i}!}\right)=\sum_{i=1}^{N}\left(-\exp\left(x_{i}^{\prime}\beta\right)+y_{i}x_{i}^{\prime}\beta-log\left(y_{i}!\right)\right) $

And the CMLE estimator is the solution to: $\sum_{i=1}^{N}\left(y_{i}-\exp\left(x_{i}^{\prime}\beta\right)\right)x_{i}^{\prime}=0$

Example 3

We have an AR(1) model: $y_{t}=\mu+\Theta y_{t}+\varepsilon_{t}$, $\varepsilon_{t}\sim N\left(0,\:\sigma^{2}\right)$

$y$ has conditional mean: $E\left[y\mid y_{t-1}\right]=\mu+\Theta y_{t}$, and unconditional mean: $E\left[y\right]=\frac{\mu}{1-\Theta}$

$y$ has conditional variance: $Var\left[y_{t}\mid y_{t-1}\right]=\sigma^{2}$ and unconditional variance: $Var\left[y_{t}\right]=\frac{\sigma^{2}}{1-\Theta^{2}}$

Am I right to assume that unconditional Maximum Likelihood estimation proceeds like this?

The unconditional density function for $y$ is: $f\left(y_{t}\right)=\left(\frac{1}{\sqrt{2\pi\frac{\sigma^{2}}{1-\Theta^{2}}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{t}-\frac{\mu}{1-\Theta}\right)^{2}}{\frac{\sigma^{2}}{1-\Theta^{2}}}\right)$

and then I state the log-likelihood function and derive the estimators as above.

Am I right to assume that conditional Maximum Likelihood estimation proceeds like this?

The conditional density function for $y$ is: $f\left(y_{t}\mid y_{t-1}\right)=\left(\frac{1}{\sqrt{2\pi\sigma^{2}}}\right)\exp\left(-\frac{1}{2}\frac{\left(y_{i}-\mu-\Theta y_{t-1}\right)^{2}}{\sigma^{2}}\right)$

and then I proceed by stating the log-likelihood function and deriving the estimators as above.

My question is if what I have done above is correct? I know that the CMLE is right since I condition on x. The question is if the UMLE part is correct? I am quite sure that Example 3 is correct but I am not so sure Example 1 and Example 2 are correct.

Any help would be greatly appreciated.

$\endgroup$
  • $\begingroup$ An equation like $\sum_{i=1}^{N}\left(y_{i}-\exp\left(x_{i}^{\prime}\beta\right)\right)x_{i}'=0$ can be expanded and simplified. $\endgroup$ – Glen_b Oct 1 '14 at 12:01
  • $\begingroup$ @Glen_B Thanks for pointing that out. I have corrected it. $\endgroup$ – Plissken Oct 1 '14 at 12:55
  • $\begingroup$ Unfortunately I think you have taken an unjustified step in your simplification (you can't exchange exponentiation and summation). I intended something rather more basic, like $\overline{y_{i}x_{i}'}=\overline{\exp\left(x_{i}^{\prime}\beta\right)x_{i}'}$, (though you may prefer to leave it as a sum rather than divide through by $N$) $\endgroup$ – Glen_b Oct 1 '14 at 23:53
  • $\begingroup$ Yeah that's what I thought. I see what you mean but I deleted it and kept the original. Thank you. $\endgroup$ – Plissken Oct 2 '14 at 8:54
  • $\begingroup$ I think there is a problem in example 1. From yi=x′iβ+εi, εi∼N(0,σ2), we can't obtain y ~ N(mu, sigma^2) which is followed by x is random. And Var(y) = Var(x'β) + σ2. $\endgroup$ – user141362 Dec 6 '16 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.