I'm reading this paper about mutual information in infoGAN infoGAN_paper_link and already have the code to run it. I pretty much found code for it which is fine and dandy except for the fact that I kinda don't understand some of the code in the cost function. So, I looked at the paper to dissect it for better understanding and came across some math notation that I don't understand (pic below). The usage of the notations I'm trying to figure out are how the "~" in the Expectation subscripts, "'", and "||" symbols are used.
This is what I think the notation means.
- "~" in the expectation subscript: the variable on the left of "~" can be any continuous value coming from whatever is to the right of the "~"
- "'" next to the "c" in P(c'|x): I have no clue. I thought those were symbols for derivates but that makes no sense for this equation so it's def not that.
- "||": I'm not sure. I only know of these symbols being used in Norms but that's obviously not the case here.
The actual code I was trying to figure out was this in PyTorch. It's the variational lower bound (mutual information term) in the cost (It's not the formulas in the screenshot above). However, I ran into the formulas in the screenshot first before getting to the formula calculations for the variational lower bound formula.
mutual_information_lower_bound = lambda c_true, mean, logvar: Normal(mean,logvar.exp()).log_prob(c_true).mean()
Thanks for the help!