# Which deep learning model can classify categories which are not mutually exclusive

Examples : I have a sentence in job description : "Java senior engineer in UK ".

I want to use a deep learning model to predict it as 2 categories : English and IT jobs. If i use traditional classification model, it only can predict 1 label with softmax function at last layer . Thus, i can use 2 model neural networks to predict "Yes"/"No" with both categories, but if we have more categories, it is too expensive . So do we have any deeplearning or machine learning model to predict 2 or more categories at same time ?

"Edit" : With 3 labels by traditional approach , it will be encoded by [1,0,0] but in my case, it will be encoded by [1,1,0] or [1,1,1]

Example : if we have 3 labels, and a sentences may be fit with all of these labels. So if output from softmax function is [0.45 , 0.35 , 0.2 ] we should classify it into 3 labels or 2 labels , or may be one ? the main problem when we do it is : what is good threshold to classify into 1, or 2 , or 3 labels ?

• We have to use sigmoid function instead of softmax function. It can assign multiple classes to the data points. – NITISH MAHAJAN Mar 28 at 4:53

You need to change the loss to binary crossentropy as the categorical cross entropy only gets the loss from the prediction for the positive targets. To understand this, look at the formula for the categorical crossentropy loss for one example $i$ (class indices are $j$):
$L_i = - \sum_j{t_{i,j} \log(p_{i,j})}$
In the normal multiclass setting, you use a softmax, so that the prediction for the correct class is directly dependent on the predictions for the other classes. If you replace the softmax by sigmoid this is no longer true, so negative examples (where $t_{i,j}=0$) are no longer used in the training! That's why you need to change to binary crossentropy, which uses both positive and negative examples: $L_i=-\sum_j{t_{i,j} \log(p_{i,j})} -\sum_j{(1 - t_{i,j}) \log(1 - p_{i,j})}$