MLE for a mixture of betas without using EM algorithm

Question

Suppose I have a mixture of two Beta densities say $f_1 = \text{Beta}(1,1)$ and $f_2= \text{Beta}(1,\beta)$ where $\beta$ is unknown. The sample $X_1,....,X_n$ is observed based on latent Bernoulli variables (unobserved) $Z_1,...,Z_n$ such that $X_i|Z_i=0 \sim f_1$ and $X_i|Z_i=1 \sim f_2$. If $P(Z_i=1)=\alpha$ (unknown). How should I find the MLE of $\alpha,\beta$?

One way to go about this is to implement an EM algorithm and get the estimates of $\alpha,\beta$. But what could be another approach to this?

Answer 1

The log of the likelihood is

$$\log(L)=\sum _{i=1}^n \log \left(1-\alpha +\frac{\alpha (1-x_i)^{\beta -1}}{B(1,\beta )}\right)$$

and you maximize that to obtain the maximum likelihood estimates. But you'll also need to obtain an estimate of precision for each parameter estimator (along with an estimate of the covariance of the two parameter estimators). Here is code using R:

# Generate data
  n <- 10000
  a <- 0.6 # alpha
  b <- 4   # beta
  set.seed(12345)
  y1 <- runif(n)
  y2 <- rbeta(n, 1, b)
  z <- rbinom(n, 1, a)
  x <- (1-z)*y1 + z*y2
  
# Log of likelihood function
  logL <- function(parms, x=x) {
    a <- parms[1]
    b <- parms[2]
    sum(log(1 - a + a*(1-x)^(b-1) / beta(1, b)))    
  }
  logL(c(0.5, 3), x=x)

# Find maximum of log of the likelihood
  mle <- optim(c(0.6, 4), logL, x=x, method="L-BFGS-B", 
    lower=c(0, 0), upper=c(1,Inf), control=list(fnscale=-1),
    hessian=TRUE)
  mle$par
  #[1] 0.6065559 3.9964919

# Now find an estimate of the covariance matrix
  covmat=-solve(mle$hessian)
#               [,1]          [,2]
# [1,]  0.0001276929 -0.0007138556
# [2,] -0.0007138556  0.0099189948

# Estimates of standard errors for alpha and beta
  sqrt(diag(covmat))
# [1] 0.01130013 0.09959415

I don't believe there is a nice closed-form solution for the maximum likelihood estimates.