4
$\begingroup$

I have a joint distribution of $(X,Y)$ where $Y$ is Bernoulli with $P(Y=1)=p=1-P(Y=0)$. The conditional distribution of $X$ given $Y=y$ is Normal with mean $\mu_y$ and variance $\sigma^2_y$: that is, a different mean and variance for each group.

They (therefore) have the following joint density:

$$f(x,y)=p^y(1-p)^{1-y}\frac{1}{\sqrt{2\pi\sigma^2_y}}\exp\left(-\frac{1}{2 \sigma^2_y}(x-\mu_y)^2\right).$$

How do I find the MLE for this model? The MLE of the $p$ parameter should just be the MLE of Bernoulli right?

Edit:

So I can see now that y is observed, so we have for instance:

$A = \{i \in \{1, \ldots, n\} \mid y_i = 0\}$

$B = \{i \in \{1, \ldots, n\} \mid y_i = 1\}$

logLikelihood: $$l(\theta)=l(p,\mu_0,\mu_1,\sigma_0,\sigma_1)=\sum_{i=1}^n\log f(x_i,y_i)=\sum_{i \in A}\log f(x_i,y_i)+\sum_{i \in B}\log f(x_i,y_i)$$ Differentiate e.g. w.r.t. $\mu_0$: $$\frac{\partial l(\theta)}{\partial \mu_0}=\frac{1}{\sigma^2_0}\sum_{i \in A}(x_i-\mu_0)\implies \hat{\mu}_0=\frac{1}{n_A}\sum_{i \in A}x_i$$

$\endgroup$
5
  • $\begingroup$ It depends if you assume $y$ is observed or not. $\endgroup$
    – Duffau
    Apr 5, 2016 at 21:31
  • $\begingroup$ I'm not sure, but does it even make sense if y is not observed? I'm thinking that maybe the MLE for mean and variance here are simply given by the usual formula but only using the relevant data ie. all x with y=0. $\endgroup$
    – ChuckP
    Apr 5, 2016 at 21:47
  • 1
    $\begingroup$ So yes it does make sense to if y is not observed, it's just not possible to compute the exact likelihood value. But you can still have a consistent estimator of the 5 parameters by using the EM-algorithm. $\endgroup$
    – Duffau
    Apr 5, 2016 at 21:51
  • 1
    $\begingroup$ Note that this is a very special case of ordinary least squares regression of $X$ against $Y$ with intercept $\mu_0$ and slope $\mu_1-\mu_0$. Thus all the usual theory and formulas apply without any change. $\endgroup$
    – whuber
    Apr 5, 2016 at 23:14
  • $\begingroup$ I tried to apply the formula in my edit, can you see if I'm on the right track? $\endgroup$
    – ChuckP
    Apr 6, 2016 at 0:07

1 Answer 1

2
$\begingroup$

Yes. Everything to the right of $p^y (1 - p)^{1 - y}$ is only a constant factor, so as a function of $p$ this whole expression is proportional to the Bernoulli likelihood.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.