3
$\begingroup$

In most of deep learning classification problem settings, the predictions $p(y|x)$ are often modeled to be independent, namely, $$p(y|x) = \prod_{i=1}^{N} p(y_i|x_i)$$ However, this assumption may violate our prior knowledge that the labels could be highly correlated. For example, in face attribute classification, the hair color could be highly correlated to the eye color, and the above model lacks the ability to infer the correlation between predictions, such as $p(y_i, y_j|x)$ or $I(y_i;y_j|x)$.

I would like to know if there is any literature or papers discussing modeling the correlation between predictions given only the dataset $\{x_i, y_i\}_{i=1}^N$. Or, if necessary, given certain additional information is it possible to model the "conditional" correlation $p(y_i, y_j|x)$?

Any input would be greatly appreciated.

$\endgroup$
1
  • $\begingroup$ any idea? any inputs related to this question is highly welcome. $\endgroup$
    – hpwww
    Oct 15, 2019 at 8:12

1 Answer 1

0
$\begingroup$

Actually it's in evaluation mode at this point of time.

Early solutions: People used to transfer learning, where once they learnt good model for one task(hairline colour), they use the weights of the initial layers of that model(hairline colour model) to another model for new task(eye colour). Here they use previous model weights for initialisation problem only, which will help in converging the training quickly. And thereafter, both models grow independently from there onwards, still this is one of the predominant solution in production.

Current state of art solution: Multi head training is getting popular in research, where they share common input and first few hidden layers, then at the end they diverge to their own layers to learn specific tasks, this way when you are learning for the hair line color task the weights learnt (through back propagation) will be helpful for learning eye line color as well and vice versa, together they improve the model since we are back propagating on n correlated tasks.

The example architecture for your problem would look like below (Note that we can have any type and any number of layers in common stack / specific task stack). Here task 1 is eye line colour, task 2 is hair line colour, and task 3 could be like eyes shape...

enter image description here

$\endgroup$
4
  • $\begingroup$ Thanks for you answer, but I am not saying multi-task learning or transfer learning. I want some modification to existing classification models such that we can learn the prediction correlation from data itself. $\endgroup$
    – hpwww
    Oct 10, 2019 at 12:18
  • $\begingroup$ I just want to clear that, here task 1 and task 2 are correlated labels like hair colour and eye colour, if you observe the lower layers of CNN they learn about the different shapes and intensities of the shapes (lower level layer learns edges with orientation, deeper layers have geometry shapes and even deeper learns the complex patterns of facial attributes). <Follow next comment> $\endgroup$
    – yugandhar
    Oct 10, 2019 at 16:45
  • $\begingroup$ <Continuing> Since eye colour and hair line colours are just function of these patterns it is learning correlated features together. If you give uncorrelated features it's not able to learn such tasks together. There is no way for computer to say that hair line colour and eye line colour are correlated. Usually one label can be given as output to another model to predict another facial attribute, but it turns out to inefficient to train multiple models, and difficult to choose which one should be trained first. $\endgroup$
    – yugandhar
    Oct 10, 2019 at 16:45
  • $\begingroup$ I probably didn't make it clear. What you have done is just a variant of classical classification models. Despite it, you still don't know the correlation between "hair color" and "eye color". $\endgroup$
    – hpwww
    Oct 11, 2019 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.