Suppose I have a mixture of two Beta densities say $f_1 = \text{Beta}(1,1)$ and $f_2= \text{Beta}(1,\beta)$ where $\beta$ is unknown. The sample $X_1,....,X_n$ is observed based on latent Bernoulli variables (unobserved) $Z_1,...,Z_n$ such that $X_i|Z_i=0 \sim f_1$ and $X_i|Z_i=1 \sim f_2$. If $P(Z_i=1)=\alpha$ (unknown). How should I find the MLE of $\alpha,\beta$?

One way to go about this is to implement an EM algorithm and get the estimates of $\alpha,\beta$. But what could be another approach to this?


1 Answer 1


The log of the likelihood is

$$\log(L)=\sum _{i=1}^n \log \left(1-\alpha +\frac{\alpha (1-x_i)^{\beta -1}}{B(1,\beta )}\right)$$

and you maximize that to obtain the maximum likelihood estimates. But you'll also need to obtain an estimate of precision for each parameter estimator (along with an estimate of the covariance of the two parameter estimators). Here is code using R:

# Generate data
  n <- 10000
  a <- 0.6 # alpha
  b <- 4   # beta
  y1 <- runif(n)
  y2 <- rbeta(n, 1, b)
  z <- rbinom(n, 1, a)
  x <- (1-z)*y1 + z*y2
# Log of likelihood function
  logL <- function(parms, x=x) {
    a <- parms[1]
    b <- parms[2]
    sum(log(1 - a + a*(1-x)^(b-1) / beta(1, b)))    
  logL(c(0.5, 3), x=x)

# Find maximum of log of the likelihood
  mle <- optim(c(0.6, 4), logL, x=x, method="L-BFGS-B", 
    lower=c(0, 0), upper=c(1,Inf), control=list(fnscale=-1),
  #[1] 0.6065559 3.9964919

# Now find an estimate of the covariance matrix
#               [,1]          [,2]
# [1,]  0.0001276929 -0.0007138556
# [2,] -0.0007138556  0.0099189948

# Estimates of standard errors for alpha and beta
# [1] 0.01130013 0.09959415

I don't believe there is a nice closed-form solution for the maximum likelihood estimates.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.