The terms Contrastive Divergence and Importance Sampling are sometimes used interchangeably.

I understand that both are used to approximate partition functions (normalization terms for probabilities) but what, if any, is the difference between them?


Importance sampling is more generic, i.e., a general technique for estimating properties of a particular distribution, while only having samples generated from a different distribution than the distribution of interest. It is related to umbrella sampling in computational physics. Depending on the application, the term may refer to the process of sampling from this alternative distribution, the process of inference, or both.

Contrastive divergence Although "unrestricted Boltzmann machines may have connections between hidden units," a restricted Boltzmann machine is more particular and applied in the more narrow context of no connections between hidden cells and "...allows for more efficient training algorithms than are available for the general class of Boltzmann machines, in particular the gradient-based contrastive divergence algorithm."

| cite | improve this answer | |
  • $\begingroup$ did you mean to say: narrow context of "restricted" Boltzmann machines which do not have connections between hidden units ? $\endgroup$ – Felipe Almeida Feb 19 '17 at 7:24
  • 1
    $\begingroup$ @FelipeAlmeida Well, yes. I copied and pasted without the prior modifying sentence, now corrected above. $\endgroup$ – Carl Feb 20 '17 at 20:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.