I am using Tensorflow for multi-label classification of Audio. The dataset I am using is made up of 10 different classes, and to each sample of audio correspond two labels. In other words, the number of overlapping sounds/classes per file is always 2. So the vector of labels "y" would be for example:
y = [0, 0, 0, 0, 0, 1, 0, 1, 0, 0]
I am using 3 convolutional layers , 1 FC layer and 1 output layer made up of 10 logistic units. And as cost function I am using the binary cross entropy for each unit and then averaging the result.
I don't know if this actually has any consequences but all my inputs are negative values.
For some reason(s), I guess mainly as a consequence of the initialization of the parameters (weights) of the network and the logistic units in the output layer, the network tends to predict mainly zeros. I guess after some training this trend will start to dissapear, however, I wonder if there is a smart way of doing something so the network does not predict those many zeros and predicts more ones, which I believe would make the cost higher and the gradient stronger (not 100% sure about this one though) and therefore training would be faster.
I use Xavier initialization for the weights of all layers, as an example:
W2 = tf.get_variable("W", [5,5,24,48], initializer=tf.contrib.layers.xavier_initializer())