I was reading this paper Online Bounds for Bayesian algorithms and they had some derivations. I didn't get how they arrived to the conclusion
I didn't get how equation 3 was derived any suggestions guys?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Using only the definition of $f_y(z)$ and single-variable Calculus: $$ \begin{align} f_y(z) &= -\log\left\{ \frac{1}{\sqrt{2\pi \sigma^2}} \exp \frac{-(z - y)^2}{2\sigma^2} \right\}\\ & = -\log\left\{ \frac{1}{\sqrt{2\pi \sigma^2}} \right\} - \frac{(z - y)^2}{2\sigma^2} \\ \implies \frac{d}{dz}f_y(z) &= 0 - \frac{2(z - y)}{2\sigma^2} \times \frac{d}{dz}(z-y)\\ &= -\frac{(z - y)}{\sigma^2} \\ \implies \frac{d^2}{dz^2}f_y(z) &= -\frac{1}{\sigma^2}. \end{align} $$
For the latter, one can get that $$ f''_{y=1}(z) = -\frac{e^z}{(1+e^z)^2} = -\frac{1}{e^{-z} + 2 + e^{z}}. $$ Since $e^x > 0 $ for $x\in\mathbb{R}$ $$ e^z, e^{-z} > 0 \implies e^{-z} + 2 + e^{z} > 1 \implies \frac{1}{e^{-z} + 2 + e^{z}} < 1 \implies |f''_{y=1}(z)| < 1. $$ The case $y = 0$ follows similarly.
