In Restricted Boltzman Machines, when collecting the statistics I sometimes heard of fantasy particles being used.
What are these? How are they useful?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
Fantasy particles were first introduced by Tielman, using a technique for training RBMs called persistent contrast divergence. See:
A more radical departure from CD1 is called “persistent contrastive divergence” (Tieleman, 2008). Instead of initializing each alternating Gibbs Markov chain at a datavector, which is the essence of CD learning, we keep track of the states of a number of persistent chains or “fantasy particles”. Each persisitent chain has its hidden and visible states updated one (or a few) times after each weight update. The learning signal is then the difference between the pairwise statistics measured on a mini- batch of data and the pairwise statistics measured on the persistent chains. Typically the number of persistent chains is the same as the size of a mini-batch, but there is no good reason for this. The persistent chains mix surprisingly fast because the weight-updates repel each chain from its current state by raising the energy of that state (Tieleman and Hinton, 2009).