Finding the MLE for a mixture of random variables which are discrete and continuous I came up with the following situation:
I have n i.i.d: $X_i \sim U(0,1)$ and another $Y_i = I_{(X_i < p)}$ (where $0<p<1$). Now obviously $Y_i \sim \mathrm{Bern}(p) $.
I want to find the MLE for $p$, using both $X_i$ and $Y_i$.
My intuition tells me there is information in the nearest $X_i$ around $p$, and also (obviously) in $\sum Y_i$.
Is there a way to define the joint probability (or density) of $X$ and $Y$ together ($P_{X,Y}(x,y)$ / $f_{X,Y}(x,y)$) so to get and maximize the likelihood?
Thanks.
 A: You are implicitly assuming the $(X_i,Y_i)$ are iid.  Therefore you may freely re-index the observations $(x_i,y_i)$ so that $x_0 = 0 \le x_1 \le x_2 \cdots \le x_n \le 1 = x_{n+1}$.  The definition of $Y_i$ implies there exists an index $k$ for which 
$$y_1 = y_2 = \cdots = y_k = 1;\ y_{k+1}=y_{k+2}=\cdots=y_n = 0.$$
When $p$ is such that $x_k \le p \le x_{k+1}$ the likelihood is nonzero and equals
$$L(p) = p^k(1-p)^{n-k}.$$
For any other value of $p$ the likelihood is zero, demonstrating we may confine the search for a maximum to the interval $[x_k, x_{k+1}]$.  Within the interior of this interval the log likelihood
$$\Lambda(p) = k\log(p) + (n-k)\log(1-p)$$
has derivative
$$\frac{d\Lambda}{dp}(p) = \frac{k}{p} - \frac{n-k}{1-p}$$
which (as a function of the interval $(0,1)$) is positive for small $p$, negative for large $p$, and zero where $p=k/n$.  This leads to three circumstances:


*

*When $x_k \lt k/n \lt x_{k+1}$, then $\hat p = k/n$.  Moreover, $\Lambda$ is smooth in a neighborhood of $\hat p$ (implying the usual Hessian/Fisher Information/score techniques apply for large $n$).

*When $k/n \le x_k$, then $\hat p = x_k$.  However, $\Lambda$ is discontinuous at this value, so the usual MLE estimates of standard errors, confidence intervals, etc do not apply.

*When $k/n \ge x_{k+1}$, then $\hat p = x_{k+1}$.  The same caution applies as in (2). 

It might be of interest to compute the chances of these three cases.  In (1), exactly $k$ of the $n$ $x_i$ are in the interval $[0, p]$ and $n-k$ are in its complement.  The chance of this Binomial event is $\binom{n}{k}p^k(1-p)^{n-k}$.  This chance approaches zero asymptotically (at a $O(n^{-1/2})$ rate).  Thus for large $n$ we can expect that case (1) rarely holds.
