Timeline for KL divergence invariant to affine transformation?
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 3, 2021 at 16:30 | comment | added | Joram Soch | @shimao: Thanks, I have added your proof here. | |
| Apr 23, 2018 at 1:33 | comment | added | shimao | @me_Tchaikovsky yes, i think that is correct | |
| Apr 22, 2018 at 3:12 | comment | added | meTchaikovsky | @shimao from the proof, I think KL divergence is invariant to transformations which have an inverse which is differentiable, is it correct? | |
| Apr 22, 2018 at 2:27 | vote | accept | meTchaikovsky | ||
| Apr 22, 2018 at 1:45 | comment | added | meTchaikovsky | @shimao I found my mistake, I forget that I'm integrating over a new distribution so that I mistook $\int dx' P_1(x')(x'-\mu_1)^2 = \sigma_1^2$, which should be $\frac{\sigma_1^2}{m^2}$ | |
| Apr 22, 2018 at 1:31 | comment | added | meTchaikovsky | @ jbowman sorry, I haven't made myself clear, what I meant is that I have $P_2(x') = P_2(x)\frac{dx}{dx'} = \sigma P_2(x)$. Since we have the inverse of the affine transformation $x = \mu_1 + \sigma (x' - \mu_1)$, then I substitute the $x$ in $\sigma P_2(x)$ by $x'$, thus I have $P_2(x')$ that is represented by $x'$. | |
| Apr 22, 2018 at 1:19 | comment | added | jbowman | You can't have it that $P_2(x')$ depends on $x'$ but not $x$ and also that $x$ is an affine transformation of $x'$. | |
| Apr 21, 2018 at 22:31 | comment | added | meTchaikovsky | I cannot see what is wrong with the substitution that I did. From $P_2(x')dx' = P_2(x)dx $, I have $P_2(x') = P_2(x)\sigma$ which is exactly what you pointed out. However, $P_2(x')$ is dependent of $x'$ but not $x$, so I substitute $x$ with $x'$ by $x = \mu_1+\sigma(x'-\mu_1)$, so $P_2(x')$ will be that normal distribution. What is wrong with this substitution? | |
| Apr 21, 2018 at 15:43 | history | answered | shimao | CC BY-SA 3.0 |