3
$\begingroup$

I have a joint distribution of $(X,Y)$ where $Y$ is Bernoulli with $P(Y=1)=p=1-P(Y=0)$. The conditional distribution of $X$ given $Y=y$ is Normal with mean $\mu_y$ and variance $\sigma^2_y$: that is, a different mean and variance for each group.

They (therefore) have the following joint density:

$$f(x,y)=p^y(1-p)^{1-y}\frac{1}{\sqrt{2\pi\sigma^2_y}}\exp\left(-\frac{1}{2 \sigma^2_y}(x-\mu_y)^2\right).$$

How do I find the MLE for this model? The MLE of the $p$ parameter should just be the MLE of Bernoulli right?

Edit:

So I can see now that y is observed, so we have for instance:

$A = \{i \in \{1, \ldots, n\} \mid y_i = 0\}$

$B = \{i \in \{1, \ldots, n\} \mid y_i = 1\}$

logLikelihood: $$l(\theta)=l(p,\mu_0,\mu_1,\sigma_0,\sigma_1)=\sum_{i=1}^n\log f(x_i,y_i)=\sum_{i \in A}\log f(x_i,y_i)+\sum_{i \in B}\log f(x_i,y_i)$$ Differentiate e.g. w.r.t. $\mu_0$: $$\frac{\partial l(\theta)}{\partial \mu_0}=\frac{1}{\sigma^2_0}\sum_{i \in A}(x_i-\mu_0)\implies \hat{\mu}_0=\frac{1}{n_A}\sum_{i \in A}x_i$$

$\endgroup$
  • $\begingroup$ It depends if you assume $y$ is observed or not. $\endgroup$ – Duffau Apr 5 '16 at 21:31
  • $\begingroup$ I'm not sure, but does it even make sense if y is not observed? I'm thinking that maybe the MLE for mean and variance here are simply given by the usual formula but only using the relevant data ie. all x with y=0. $\endgroup$ – ChuckP Apr 5 '16 at 21:47
  • 1
    $\begingroup$ So yes it does make sense to if y is not observed, it's just not possible to compute the exact likelihood value. But you can still have a consistent estimator of the 5 parameters by using the EM-algorithm. $\endgroup$ – Duffau Apr 5 '16 at 21:51
  • 1
    $\begingroup$ Note that this is a very special case of ordinary least squares regression of $X$ against $Y$ with intercept $\mu_0$ and slope $\mu_1-\mu_0$. Thus all the usual theory and formulas apply without any change. $\endgroup$ – whuber Apr 5 '16 at 23:14
  • $\begingroup$ I tried to apply the formula in my edit, can you see if I'm on the right track? $\endgroup$ – ChuckP Apr 6 '16 at 0:07
2
$\begingroup$

Yes. Everything to the right of $p^y (1 - p)^{1 - y}$ is only a constant factor, so as a function of $p$ this whole expression is proportional to the Bernoulli likelihood.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.