I am using 3D convolutions in tensorflow. I want to initialise the weights using Xavier initialisation. Tensorflow has an implementation of Xavier initialisation for 2D convolutions, from this reference and the original paper I figured the initialisation (for uniform distribution $[-x, x]$) should be something like: $$x = \sqrt{\frac{6}{height \times width \times depth \times (\#channels_{in} + \#channels_{out})}}$$, where $height, width, depth$ are the dimensions of the convolution's patch (3 dimensional) and $\#channels_{in}, \#channels_{out}$ are the number of channels that go in and out of the convolution layer. Where this hopefully corresponds to: $$ x = \sqrt{\frac{6}{\#weightsOfLayer}} $$ Can someone please verify if my reasoning is correct, that is, does the denominator correspond the number of weights in the layer, and should I be calculating the number of weights in the layer in the first place?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.