I see there is something called Iterated Conditional Models (ICM), which is a sort of Gibbs sampling where, instead of sampling, we use the value that maximizes the conditional. That is:

for (i in 1:iter){
   x_1[i] = argmax_{x_1} p(x_1 | x_2[i-1])
   x_2[i] = argmax_{x_2} p(x_2 | x_1[i]) 

My question is: can happens if, instead of using the maximum (the MAP) we use expectations?

for (i in 1:iter){
   x_1[i] = E[x_1 | x_2[i-1]]
   x_2[i] = E[x_2 | x_1[i]]

I would like to know whether this is some known algorithm.

My motivation is that I currently do a Gibbs sampling as the E-step of a EM algorithm, and I need ${1/N}\sum_nx_1$ to update my parameter in the M-step. Thus, I guess I might just sample $x_2$ at each Gibbs iteration and then update the expected $x_1$, then sample $x_2$ given the new expectation and so on. I guess I'm missing something on why this is terribly wrong, but I don't see what.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.