Contrastive Divergence: making hidden states binary When learning a Restricted Bolzmann Machine with Contrastive Divergence $\mathbf{CD}_n$ for $n>1$ should we make the hidden states binary in the updates that are in between?
In Hinton's - 'A Practical Guide to Training Restricted Bolzmann Machines' two contradictory reccomendations are mentioned:

3.1 "It is very important to make these hidden states binary, rather than using the probabilities themselves. [...] For the last update of
the hidden units, [...] use the probability itself"
3.4 "When [the hidden units] are being driven by reconstructions, always use probabilities without sampling."

The first quote seems to say the updates in the middle should do sampling (making binary), while the second seems to say they shouldn't! Which one is correct?
 A: I implemented it that way: 


*

*Sampling hidden from the real data, the data should be binary,
therefore, it is not an issue, we take binary. It is the very first
step of CD. 

*Sampling visible from hidden: Take the binary values (sampled values) of the hidden units. 

*Sampling hidden from visible: Take the activation probabilities of the visible units. 


Step 2 and 3 are repeated if you use more than CD-1. 
The quote "For the last update of the hidden units, [...] use the probability itself" means that for the very last update of the hidden units, you don't have to sample the values because it won't be used after, it is for efficiency. 
A: I have myself come to a very similar question, and I think i do now understand whats going on. I'll try to explain... :) I found [1] quite helpfull to come to my conclusions. I also want to stress, that I'm not entirely sure about the stuff that follows, so please correct me if I'm wrong.
Assume you do the followin sampling:
$$\mathbf{v^0} \rightarrow \mathbf{h^0} \rightarrow \mathbf{v^1}$$
where $\mathbf{v_0}$ is a visible binary data vector from your training set $\mathcal{T}$, $\mathbf{h^0}$ is the first sampled hidden binary vector and $\mathbf{v^1}$ is the first sampled visible vector (sampled from binary $\mathbf{h^0}$ and NOT its activations).
For a parameter update $\Delta w$ (taking CD1) of the weight matrix you usually write:
$$ \Delta w \propto \left<\mathbf{h^0}\mathbf{v^0}\right>_\mathcal{T} - \left<\mathbf{h^0}\mathbf{v^1}\right>_\mathcal{T}$$
In [1] they rewrite the first bracket $\left<\mathbf{h^0}\mathbf{v^0}\right>_\mathcal{T}$ as $\left<\left<\mathbf{h^0}\mathbf{v^0}\right>_{p(\mathbf{h^0}|\mathbf{v^0},w)}\right>_\mathcal{T}$. The inner brackets average over the distribution of sampled hidden vectors for a specific data vector $p(\mathbf{h^0}|\mathbf{v^0},w)$ from the training set. This distribution is given by the activation probabilities for sampling $\mathbf{h_0}$ (lets call them $\mathbf{a_h^0}$) , hence you can directly perform the inner average by taking $\mathbf{a_h^0} \mathbf{v^0}$.
I think for the second bracket, one can write:
$$ \left<\mathbf{h^0}\mathbf{v^1}\right>_\mathcal{T} = \left<\left<\left<\mathbf{h^0}\mathbf{v^1}\right>_{p(\mathbf{v^1}|\mathbf{h^0},w)}\right>_{p(\mathbf{h^0}|\mathbf{v^0},w)}\right>_\mathcal{T} $$.
Here we can only perform the most inner average by taking the activations of the visible layer $\mathbf{a_v^1}$. One can easyly see this from the following (onedimensional) example.
Lets assume the probability to be active (h or v equals 1) is given by a sigmoid function:
$$ P(h = 1 | v) = \mathcal{S}(v) \\ P(v = 1 | h) = \mathcal{S}(h)$$
the expectation values then read:
$$\left<h v\right>_{p(h|v)} = 1\times v\times \mathcal{S}(v) + 0 \times v \times (1-\mathcal{S}(v)) = v \times \mathcal{S}(v)$$ 
whereas for the second bracket one obtains:
$$\left<\left<h v\right>_{p(v|h)}\right>_{p(h|input)} = 
\left<h \times \mathcal{S}(h)\right>_{p(h|input)} = \mathcal{S}(1)\times \mathcal{S}(input) \neq \mathcal{S}(a_h)\times \mathcal{S}(input)$$.
The essence:


*

*For $CDn$ with $n>1$ we always have several nested expectation values.

*We can only replace the very last sampling of the Gibbs chain by their corresponding activations.
I think the above should explain

3.1 "It is very important to make these hidden states binary, rather than using the probabilities themselves. [...] For the last update of
  the hidden units, [...] use the probability itself"

When doing reconstructions, one usually does only one step of propagation ($v\rightarrow h$ or $h\rightarrow v$). Hence one can directly use the activations (i.e. probabilities) to get an expectation value of the sampling error.
As stated above, all of the above might be wrong. Therefore, I would be really glad if someone with more insight could comment on this explanation.
[1]: Carreira-Perpinan, Miguel A., and Geoffrey E. Hinton. "On contrastive divergence learning." Proceedings of the tenth international workshop on artificial intelligence and statistics. NP: Society for Artificial Intelligence and Statistics, 2005.
