Can the EM algorithm be applied to my problem? Input data set is based on a function of parameter

Question

I understand EM algorithm is often used for missing data/mixture problem. But can it be used to optimize a particular type of likelihood based on jointly fitting variables and transformations of those variables?

I am fitting a logistic regression model: $Y$ is the usual binary dependent variable, $\mathbf{X}'$ is a matrix of independent variables. However, the data $\mathbf{X}' = f_{\alpha}(\mathbf{X})$ are a function of some parameters, $\alpha$. Specifically $X_{.,j}' = \alpha_j^{X_{.,j}}$. We observe only $\mathbf{X}$ and $Y$.

This gives the linear model:

$\mbox{logit} (Y|X) = \beta_0 + \beta_1 \alpha_1 ^{X_1} + \ldots + \beta_k \alpha_k ^{X_k} $

Is there any other method that jointly estimate this model? The one that I can think of is to use a global search. Steps are:

1. Randomly generate the three alpha parameters within its definition space, fit the model to get coefficient and AUC.
2. Repeat step 1 for 1000 times.
3. Choose the alphas that gives the best AUC.

Does this approach make sense? What I am trying to do is to get the optimized parameter. Thank you.

Answer 1

The EM algorithm is not applied here because there is no expectation step necessary to maximize the likelihood for the logistic model above. If doing this in R, consider using nlm, i.e. straightforward maximum likelihood can jointly estimate the vector of $\beta$ and $\alpha$ in your problem.

loglik <- function(params) {
  dbinom(params[1:k] %*% params[{k+1}:2k]^X, log=TRUE)
}

If the goal is maximizing the AUC, then you ought to consider an alternate estimation strategy based on ROC regression, though this can be very perilous because it doesn't have nearly the same robust flexibility as maximum likelihood does.