1
$\begingroup$

I am using 3D convolutions in tensorflow. I want to initialise the weights using Xavier initialisation. Tensorflow has an implementation of Xavier initialisation for 2D convolutions, from this reference and the original paper I figured the initialisation (for uniform distribution $[-x, x]$) should be something like: $$x = \sqrt{\frac{6}{height \times width \times depth \times (\#channels_{in} + \#channels_{out})}}$$, where $height, width, depth$ are the dimensions of the convolution's patch (3 dimensional) and $\#channels_{in}, \#channels_{out}$ are the number of channels that go in and out of the convolution layer. Where this hopefully corresponds to: $$ x = \sqrt{\frac{6}{\#weightsOfLayer}} $$ Can someone please verify if my reasoning is correct, that is, does the denominator correspond the number of weights in the layer, and should I be calculating the number of weights in the layer in the first place?

Many thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.