# Accelerating multi-label classification using NNs

I am using Tensorflow for multi-label classification of Audio. The dataset I am using is made up of 10 different classes, and to each sample of audio correspond two labels. In other words, the number of overlapping sounds/classes per file is always 2. So the vector of labels "y" would be for example:

y = [0, 0, 0, 0, 0, 1, 0, 1, 0, 0]

I am using 3 convolutional layers , 1 FC layer and 1 output layer made up of 10 logistic units. And as cost function I am using the binary cross entropy for each unit and then averaging the result.

I don't know if this actually has any consequences but all my inputs are negative values.

For some reason(s), I guess mainly as a consequence of the initialization of the parameters (weights) of the network and the logistic units in the output layer, the network tends to predict mainly zeros. I guess after some training this trend will start to dissapear, however, I wonder if there is a smart way of doing something so the network does not predict those many zeros and predicts more ones, which I believe would make the cost higher and the gradient stronger (not 100% sure about this one though) and therefore training would be faster.

I use Xavier initialization for the weights of all layers, as an example:

W2 = tf.get_variable("W", [5,5,24,48], initializer=tf.contrib.layers.xavier_initializer())


Maybe using tf.truncated_normal_initializer instead could help?

I use sigmoid when there are an arbitrary number of possible labels. In your case, you know you have exactly two labels. I would instead use softmax and divide the true label by two, for example [0,...,0,.5,.5,0,...].