1
$\begingroup$

I've successfully implemented a RBM and trained it with CD-k, but I now have some issues doing the same with Convolutional RBM.

The visible and hidden biases gradients are easy to compute: v1 - v2 for visible bias and h1_k - h2_k for the hidden bias. But I failed to see how to compute the gradients for the weights. The weights are NH x NH matrices. In RBM it was easy since the weight matrix was NH * NV dimension and with multiplication of h1 * v1 - h2 * v2 we could have the gradients, but the dimension are not equivalent in CRBM.

What is the rule to compute the gradients ? Or is there a different way to update the weights ? It's probably a convolution, but I fail to see how...

$\endgroup$
1
$\begingroup$

The gradients can be computed from this formula:

$$ W_{pos} = V_1 *_v \tilde{H_1} \\ W_{neg} = V_2 *_v \tilde{H_2} \\ W_{grad} = W_{pos} - W_{neg} $$

$*_v$ being a "valid* convolution. $\tilde{H}$ being $H$ flipped horizontally and vertically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.