Why do we want (Restricted) Boltzmann Machines to be stochastic? Boltzmann machines are stochastic recurrent neural networks with a specific architecture, and they are interesting in several contexts.
They are stochastic in the sense that the sampling procedure involves repeatedly computing the activation functions for the next layer (hidden from visible or visible from hidden), sampling from the result, and using the obtained sample (as opposite to directly using the activation functions) for the next iteration.
I do wonder though, why is the stochasticity useful here?
If we want to compute the feature vector corresponding to a given input, do we use a sample from the hidden neurons' activations, or do we use directly the activations?
Using the same architecture as the RBM, but with deterministic neurons, would seem to also fit more naturally with the probabilistic energy-based interpretation: the activations on the visible layer after a pass through the hidden layer would directly correspond to a "proper" conditional probability of getting that output given the input, as opposite to just being the conditional probability of that output given a sample obtained from the conditional probability over the hidden layer given the used input.
The stochasticity seems to also make the training kind of weird. If the neurons are taken to be deterministic wouldn't the whole architecture effectively correspond to a regular feedforward NN with a given number of hidden layers (and some relations between the weights and biases of the layers), which could be trained using standard backpropagation?
 A: Stochastic component in RBM (such as sampling from Boltzmann distribution) is used to find the optimal parameters of the RBM.
RBM itself is used to extract higher level features. You pre-train an RBM layer, then plug into the deep network. You train RBM in an unsupervised manner, somewhat similar to PCA or autoencoders. 
For instance, AI people are used to think of PCA as a dimensionality reduction tool, which it is. However, it can be thought of a feature extractor too. Consider, interest rates on the market. You can borrow for 1 year or 10 years. If you go the market there would be quotes for borrowing to anything from overnight to 100 years, and each will have its own interest rate. It turns out that if you run these rates through PCA and look at first three components then something like this comes out:

So, the unsupervised (dumb) PCA was able to extract three features that are interpreted as level, slope and the curvature of the interest rate yield curve. These features are widely used in finance.
So, RBM serves a similar purpose in the neural net. It's pre-trained separately to extract some features from the input, then these features are passed along to the next layer in the network.
