Context: Using a CNN to localise a object in an image. There are two kinds of objects present represented by classes C1 and C2. The output of the CNN is 6 nodes i.e. C1, C2, x, y, w, h. Where [C1,C2] = [0,1] if the class is C2 and it is [1,0] if the class is C1. x, y represent the centre of the bounding box surrounding the object and w,h represent the width and height of the bounding box.
Problem: Now I have been trying to compute softmax cross entropy loss classification ( i.e. on C1, C2 nodes ) and using L2 loss on the x,y,w and h nodes. The issue that I am facing is that one loss dominates the other loss and giving them weights to balance out each others' effect is not working very effectively. Can anyone suggest a good loss function that takes both classification and localisation into account.
Note: 1. There is an object present at all times in the image. 2. I have tried the yolo loss ( and its not good enough ) and am looking at different loss functions which people might have found useful for this kind of application.