Why are RBMs symmetric? I'm doing some experiments with RBMs and note that they use symmetric weights between the input and hidden layer.
Why is this?
I'm particularly interested in the intuition behind this design decision - why would asymmetric weights not work for example?
 A: Well, RBM is an energy based model, and as such it has undirected edges, and thus you could say "symmetric weights".
The probability distribution over visible and hidden units, defined by the RBM is based on the Energy function:
$$E = -\sum_{i,j} w_{ij} \, v_i \, h_j -\sum_i \alpha_i \, v_i - \sum_i \beta_i \, h_i$$
As you can see, even if you wanted to somehow introduce asymmetric weights, they would average out.
In short, usage of asymmetric weights simply makes no sense in case of RBM, since it is an energy-based model defined by an undirected graph.
Now, you wanted to know "What is the intuition behind this design decision". I guess you could ask this question here, "Why make RBM's energy based models defined by an undirected graph? Why not use a directed graph?". And it would be a damn deep question.
The short answer is: you can. A model similar to RBM with directed egdes is called sigmoid belief net. They are directed graphs, and not energy based. They are different in how they are trained, and in where the problems with training arises. Since it's not directly connected to your original question, and I just thought you might be interested, I'll drop you great learning material for both RBM and sigmoid belief nets:
https://class.coursera.org/neuralnets-2012-001/lecture/index
The class is taught by Geoffrey Hinton himself. I highly recommend it if you are interested in neural networks in general. Also, it might be a good idea to download the videos now, since class closes in few weeks, and then they won't be available anymore. The lectures most relevant to your question, that will also really make your understanding of RBM much deeper are 11, 12, 13, 14.
A: Back propagation Neural Network works in a "inductive/causal" way, that is, the i-th layer induces the (i+1)-th layer. It is one way directional, not bi-directional. As a result, we get a "deterministic" result rather than a stochastic result.

On the other hand, the RBM, as said, is energy based. The transition is bi-directional. That is, the i-th layer can affect (i+1)-th layer, and the (i+1)-th layer can affect the i-th layer as well. 
In such a "bi-directional" network, intuition tells us that "symmetric" network weights provides great potential benefits.
"Symmetric" means the propagation weights from the i-th layer to the (i+1)-th layer is the same as the propagation weights from the (i+1)-th layer to the i-th layer in a RBM.
i ----> (i+1) equal i <---- (i+1) 
Why it is symmetric? I guess... a symmetric network has a good chance to be stable. If not symmetric, with two different sets of weights on the left direction from the right direction, the network may behave as unstable as a "ping-pong" game, back and forth and back and forth and... explode.
Further, again I guess, if it is asymmetric, then even if the network reaches some equilibrium, the energy distribution may not be Boltzmann distribution, and we should not call it Boltzmann machine anymore. 
With symmetric weights on both directions, (and if we give the network long enough time/iterations, no matter what the initialized h and v),  the RBM network can reach an equilibrium. The equilibrium does not mean that v and h are fixed binary vectors, but it means that v and h will have a fixed probability to be binary vectors.
There are a number of states in one equilibrium. 
Each state corresponds to a probability. Each state corresponds to an energy of the whole network. We are interested in making the network to reach the equilibrium of an energy as small as possible.
For example, say v= 1 bit, h= 1 bit, we have 4 combinations, vh ={00, 01, 10, 11}, then on an equilibrium, we have fixed probability of 
Prob(vh=00) state 00
Prob(vh=01) state 01
Prob(vh=10) state 10
Prob(vh=11) state 11
and of course Prob(vh=00)+Prob(vh=01)+Prob(vh=10)+Prob(vh=11)=1
The Probs are defined by the definition of RBM, apparently.
$$
Prob(v,h) = \frac{e^{-E(v,h)}} {Z}
$$
where $Z$ is the sum of all possible states, $Z = \sum_{v,h} e^{-E(v,h)}$.
(refer to wiki RBM)
Note: symmetric does not mean that the weight matrix, which is noted as W in many literatures, is a symmetric matrix. NO, W is not a symmetric matrix. For one thing, a symmetric matrix is always square. However, apparently RBM weights matrix does not have to be a square matrix. That is, the number of hidden units do not have to be same as visible units. It is very misleading that many literature claim that the RBM weight matrix is "symmetric".
