I am new to KL Divergence Loss (and indeed all similar comparisons between discrete series data). The output of my network produces a series of tuples of a length that varies during training. The target data series' size is always steady.

Is there a way to perform a comparison between the two sets, despite the fact that they probably do not have the same sizes most of the time? I was planning to implement an extra loss component penalising for the size difference, but am not sure how to do the actual comparison between elements.

  • $\begingroup$ I do not think that that can or should be done. KL divergence is self-referential, without the 'self' there is no reference. $\endgroup$ – Carl Jun 15 at 2:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.