# MLE for joint distribution

I have a joint distribution of $(X,Y)$ where $Y$ is Bernoulli with $P(Y=1)=p=1-P(Y=0)$. The conditional distribution of $X$ given $Y=y$ is Normal with mean $\mu_y$ and variance $\sigma^2_y$: that is, a different mean and variance for each group.

They (therefore) have the following joint density:

$$f(x,y)=p^y(1-p)^{1-y}\frac{1}{\sqrt{2\pi\sigma^2_y}}\exp\left(-\frac{1}{2 \sigma^2_y}(x-\mu_y)^2\right).$$

How do I find the MLE for this model? The MLE of the $p$ parameter should just be the MLE of Bernoulli right?

Edit:

So I can see now that y is observed, so we have for instance:

$A = \{i \in \{1, \ldots, n\} \mid y_i = 0\}$

$B = \{i \in \{1, \ldots, n\} \mid y_i = 1\}$

logLikelihood: $$l(\theta)=l(p,\mu_0,\mu_1,\sigma_0,\sigma_1)=\sum_{i=1}^n\log f(x_i,y_i)=\sum_{i \in A}\log f(x_i,y_i)+\sum_{i \in B}\log f(x_i,y_i)$$ Differentiate e.g. w.r.t. $\mu_0$: $$\frac{\partial l(\theta)}{\partial \mu_0}=\frac{1}{\sigma^2_0}\sum_{i \in A}(x_i-\mu_0)\implies \hat{\mu}_0=\frac{1}{n_A}\sum_{i \in A}x_i$$

• It depends if you assume $y$ is observed or not. Apr 5, 2016 at 21:31
• I'm not sure, but does it even make sense if y is not observed? I'm thinking that maybe the MLE for mean and variance here are simply given by the usual formula but only using the relevant data ie. all x with y=0. Apr 5, 2016 at 21:47
• So yes it does make sense to if y is not observed, it's just not possible to compute the exact likelihood value. But you can still have a consistent estimator of the 5 parameters by using the EM-algorithm. Apr 5, 2016 at 21:51
• Note that this is a very special case of ordinary least squares regression of $X$ against $Y$ with intercept $\mu_0$ and slope $\mu_1-\mu_0$. Thus all the usual theory and formulas apply without any change.
– whuber
Apr 5, 2016 at 23:14
• I tried to apply the formula in my edit, can you see if I'm on the right track? Apr 6, 2016 at 0:07

Yes. Everything to the right of $p^y (1 - p)^{1 - y}$ is only a constant factor, so as a function of $p$ this whole expression is proportional to the Bernoulli likelihood.