# Fisher Information Inequality of a function of a random variable

Suppose I have a random variable $X \sim f_{X}(x \mid \lambda)$ with support over $(0, \infty)$ and I find the Fisher information in $X$ about $\lambda$, i.e., $$I_{X}(\lambda)=\mathbb{E}\left[\left(\dfrac{\partial\ell_X}{\partial\lambda}\right)^2\mid\lambda \right]$$ where $\ell_X$ is the log-likelihood of $X$, which is just merely $\ell_X(\lambda) = \log f_{X}(x \mid \lambda)$.

Now let $Y = \text{floor}(X)$, i.e., the rounded-down-to-the-nearest-integer version of $X$. Can I make any claims about $I_Y(\lambda)$?

This arose in a qualifying exam solution as follows: suppose $X \sim \text{Exp}(\lambda)$, i.e., $$f_{X}(x) = \lambda e^{\lambda x}\cdot \mathbf{1}_{(0, \infty)}(x)$$ and let $Y = \text{floor}(X)$. Then $I_{X}(\lambda) = 1/\lambda^2$ and $I_{Y}(\lambda) = e^{-\lambda}/(1-e^{\lambda})^2$.

Furthermore, since $Y$ is a function of $X$, $I_{Y}(\lambda) \leq I_{X}(\lambda)$. Why is this? Is there a theorem that I don't know about?

I've tried asking about how to compute this inequality directly, but showing this isn't easy given timing on a qualifying exam, and it would be more useful if I understood why $I_{Y}(\lambda) \leq I_{X}(\lambda)$ follows from $Y$ being a function of $X$.

EDIT: I have managed to find one mention of this inequality at http://cs.stanford.edu/~ppasupat/a9online/1237.html:

For other statistics $T(X)$, $I_{T}(\theta) \leq I_{X}(\theta)$.

Alas, no proof.

• I'll try fisher information as the names suggest is a way to measure the amount of information X has on the parameter $\lambda$. Since, $Y$ is a function of $X$, $X$ must carry more information than $Y$. My guess that for all function of X that are not Injective the inequality would be sharp, and for function that are injective it would be equal (that is if $y = x + 2$ then $I_y(\lambda) = I_x(\lambda)$). – Kozolovska Apr 23 '17 at 7:07
• See Lemma 1.3.2 in stat.tamu.edu/~suhasini/teaching613/chapter1.pdf – S. Catterall Reinstate Monica Apr 28 '17 at 14:07

I am not aware of any general rule to pass over to the Fisher information of the floor function, but in this case it is possible to solve the problem for the specific distribution. The easiest way to do this is to explicitly determine the distribution of $Y = \text{floor} (X)$ and then find the Fisher information for the discrete distribution. For each possible argument value of the floor $y = 0,1,2, ...$ we have:

$$p_Y(y) = \mathbb{P}(y \leqslant X < y+1) = (1-e^{-\lambda(y+1)}) - (1-e^{-\lambda y}) = (1 - e^{-\lambda}) e^{-\lambda y}.$$

This is a discrete distribution, with expected value:

$$\mathbb{E}(Y) = \sum_{y=0}^\infty y \cdot p_Y(y) = (1 - e^{-\lambda}) \sum_{y=0}^\infty y e^{-\lambda y} = \frac{e^{-\lambda}}{1-e^{-\lambda}}.$$

This distribution has score function:

$$s_y(\lambda) \equiv \frac{\partial}{\partial \lambda} \ln p_Y(y | \lambda) = \frac{\partial}{\partial \lambda} ( \ln(1 - e^{-\lambda}) -\lambda y ) = \frac{e^{-\lambda}}{1-e^{-\lambda}} - y.$$

Since the expected score is zero, the Fisher information is given by:

\begin{aligned} I_Y(\lambda) &= \mathbb{E} \Big[ s_Y(\lambda)^2 \Big] \\[6pt] &= \sum_{y=0}^\infty \Big( \frac{e^{-\lambda}}{1-e^{-\lambda}} - y \Big)^2 p_Y(y) \\[6pt] &= \sum_{y=0}^\infty \Big( \frac{e^{-2\lambda}}{(1-e^{-\lambda})^2} - \frac{2e^{-\lambda}}{1-e^{-\lambda}} \cdot y + y^2 \Big) (1 - e^{-\lambda}) e^{-\lambda y} \\[6pt] &= \frac{e^{-2\lambda}}{1-e^{-\lambda}} \sum_{y=0}^\infty e^{-\lambda y} - 2 e^{-\lambda} \sum_{y=0}^\infty y e^{-\lambda y} + (1 - e^{-\lambda}) \sum_{y=0}^\infty y^2 e^{-\lambda y} \\[6pt] &= \frac{e^{-2\lambda}}{(1-e^{-\lambda})^2} - \frac{2 e^{-2\lambda}}{(1-e^{-\lambda})^2} + \frac{e^{-\lambda}(1+e^{-\lambda})}{(1-e^{-\lambda})^2} \\[6pt] &= \frac{1}{(1-e^{-\lambda})^2} \Bigg[e^{-2\lambda} - 2 e^{-2\lambda} + e^{-\lambda}(1+e^{-\lambda}) \Bigg] \\[6pt] &= \frac{1}{(1-e^{-\lambda})^2} \Bigg[e^{-2\lambda} - 2 e^{-2\lambda} + e^{-\lambda}+e^{-2\lambda} \Bigg] \\[6pt] &= \frac{e^{-\lambda}}{(1-e^{-\lambda})^2}. \\[6pt] \end{aligned}

We can now verify the inequality $I_Y(\lambda) \leqslant I_X(\lambda)$ directly. Using a bit of algebra, and expanding the exponentials using the Maclaurin series representation, we obtain:

\begin{aligned} I_Y(\lambda) = \frac{e^{-\lambda}}{(1-e^{-\lambda})^2} &= \frac{e^{-\lambda}}{1 - 2e^{-\lambda} + e^{-2\lambda}} \\[6pt] &= \Big( \frac{1 - 2e^{-\lambda} + e^{-2\lambda}}{e^{-\lambda}} \Big)^{-1}\\[6pt] &= (e^{\lambda} - 2 + e^{-\lambda})^{-1} \\[6pt] &= \Big( \sum_{k=0}^\infty \frac{\lambda^k}{k!} - 2 + \sum_{k=0}^\infty \frac{(-1)^k \lambda^k}{k!} \Big)^{-1} \\[6pt] &= \Big( 1 + \sum_{k=1}^\infty \frac{\lambda^k}{k!} - 2 + 1 + \sum_{k=1}^\infty \frac{(-1)^k \lambda^k}{k!} \Big)^{-1} \\[6pt] &= \Big( \sum_{k=1}^\infty \frac{\lambda^k}{k!} + \sum_{k=1}^\infty \frac{(-1)^k \lambda^k}{k!} \Big)^{-1} \\[6pt] &= \Big( 2 \sum_{k=1}^\infty \frac{\lambda^{2k}}{(2k)!} \Big)^{-1} \\[6pt] &= \Big( \sum_{k=1}^\infty \frac{\lambda^{2k}}{(2k)! / 2} \Big)^{-1} \\[6pt] &= \frac{1}{ \lambda^2 + \frac{\lambda^4}{12} + \frac{\lambda^6}{360} + \cdots} \\[6pt] &\leqslant \frac{1}{\lambda^2} = I_X(\lambda). \end{aligned}