# ReLUs improve Restricted Boltzmann Machines

This question is about the use of Rectified Linear Units as hidden units in Restricted Boltzmann Machines.

In Nair and Hinton's paper, using ReLUs as hidden units is proposed. In section 1, they discuss the use of Bernoulli and Gaussian units (nothing new here). They provide the energy of an RBM with Guassian hidden units as

$$E(\textbf{v, h}) = \sum \frac{(v_i-b_i)^2}{2\sigma_i^2} - \sum b_j h_j - \sum \frac{v_i}{\sigma_i} h_j w_{ij}.$$

Sampling from the hidden units is easy as it is easy to interpret the output of a sigmoid function as a probability.

In section 2 they introduce ReLUs as hidden units. There are two things that I don't understand

1. How should the energy function change?
2. How should max(0,x) be interpreted as a probability?