# CNN architecture for multilabel classification on audio files

I have a multilabel classification on audio files and I'm troubled about the architecture. First of all, I would like my model to output the probabilities of each label which in my case are all independent (don't need to sum up to 1).

So I have constructed a CNN that consists of :

• 3 convolutional layers
• 1 fully connected and
• output layer

Regarding the activation functions of each layer I chose ReLu for the 3 convolutional and the fully connected and sigmoid for the output. The loss function is also chosen as sigmoid_cross_entropy_with_logits (I'm using tensorflow).

The problem is that the produced output is not a probability but simply 0 or 1 and this is actually normal as ReLu outputs positive values which are not upperbounded while the sigmoid is flat for values higher than 5.

Also the weights and the bias I use as sampled from the normal distribution.

What should I do ? My thoughts so far are:

• Change the activation of the convolutional and fully connected layers so that they produce bounded values to feed into the sigmoid.
• Sample weights and biases from another distribution other than normal so that when multiplied with layer's output will give relatively small values.

Some pieces of the corresponding code:

Weights and biases initialization

weights = {
'wc1': tf.Variable(tf.random_normal([10, 10, 1, 128])),
'wc2': tf.Variable(tf.random_normal([10, 10, 128, 284])),
'wc3': tf.Variable(tf.random_normal([10, 10, 284, 768])),
'wd1': tf.Variable(tf.random_normal([10*10*768, 2048])),
'out': tf.Variable(tf.random_normal([2048, n_classes]))
}
biases = {
'bc1': tf.Variable(tf.random_normal([128])),
'bc2': tf.Variable(tf.random_normal([284])),
'bc3': tf.Variable(tf.random_normal([768])),
'bd1': tf.Variable(tf.random_normal([2048])),
'out': tf.Variable(tf.random_normal([n_classes]))
}


CNN definition

def conv_net(x, weights, biases, dropout):
x = tf.reshape(x, shape=[-1, 120, 120, 1])
# 1st Convolution Layer
conv1 = conv2d(x, weights['wc1'], biases['bc1'])
# Max Pooling (down-sampling)
conv1 = maxpool2d(conv1, k=6)

# 2nd Convolution Layer
conv2 = conv2d(conv1, weights['wc2'], biases['bc2'])
# Max Pooling (down-sampling)
conv2 = maxpool2d(conv2, k=2)

# 3rd Convolution Layer (without maxpooling)
conv3 = conv2d(conv2, weights['wc3'], biases['bc3'])

# Fully connected layer
# Reshape conv2 output to fit fully connected layer input
fc1 = tf.reshape(conv3, [-1, weights['wd1'].get_shape().as_list()[0]])
print (fc1.get_shape().as_list())
print (fc1.get_shape().as_list())
fc1 = tf.nn.relu(fc1)
print (fc1.get_shape().as_list())
# Apply Dropout
fc1 = tf.nn.dropout(fc1, dropout)

# Output, class prediction
out = tf.nn.sigmoid(out)
return out