Bias initialization in convolutional neural network What is the correct way to initialize biases in convolutional neural networks (tf.zeros, tf.truncated_normal, tf.random_normal), and why?
Should biases be constant? Do we even need biases in a deep neural network (like VGG-16)?
In a siamese neural network, do we also share the biases along with the weights?
 A: Just noting that the answer to this question suggests setting CNN biases to 0, quoting CS231n Stanford course:

Initializing the biases. It is possible and common to initialize the
  biases to be zero, since the asymmetry breaking is provided by the
  small random numbers in the weights. For ReLU non-linearities, some
  people like to use small constant value such as 0.01 for all biases
  because this ensures that all ReLU units fire in the beginning and
  therefore obtain and propagate some gradient. However, it is not clear
  if this provides a consistent improvement (in fact some results seem
  to indicate that this performs worse) and it is more common to simply
  use 0 bias initialization.

source: http://cs231n.github.io/neural-networks-2/
A: Usually you initialise them to 1.0
Biases should be trainable variables not constant, their value must be allowed to change during training. Biases are necessary in every deep network architecture I know of, without them your network will most likely be unable to learn anything.
I don't know what a siamese neural network is but in architectures where weights are shared (such convolution neural networks) weights and biases are always shared together as they come in pairs, the combination of the two is what defines a layer.
