6
$\begingroup$

Suppose I have a random variable $X \sim f_{X}(x \mid \lambda)$ with support over $(0, \infty)$ and I find the Fisher information in $X$ about $\lambda$, i.e., $$I_{X}(\lambda)=\mathbb{E}\left[\left(\dfrac{\partial\ell_X}{\partial\lambda}\right)^2\mid\lambda \right]$$ where $\ell_X$ is the log-likelihood of $X$, which is just merely $\ell_X(\lambda) = \log f_{X}(x \mid \lambda)$.

Now let $Y = \text{floor}(X)$, i.e., the rounded-down-to-the-nearest-integer version of $X$. Can I make any claims about $I_Y(\lambda)$?

This arose in a qualifying exam solution as follows: suppose $X \sim \text{Exp}(\lambda)$, i.e., $$f_{X}(x) = \lambda e^{\lambda x}\cdot \mathbf{1}_{(0, \infty)}(x)$$ and let $Y = \text{floor}(X)$. Then $I_{X}(\lambda) = 1/\lambda^2$ and $I_{Y}(\lambda) = e^{-\lambda}/(1-e^{\lambda})^2$.

Furthermore, since $Y$ is a function of $X$, $I_{Y}(\lambda) \leq I_{X}(\lambda)$. Why is this? Is there a theorem that I don't know about?

I've tried asking about how to compute this inequality directly, but showing this isn't easy given timing on a qualifying exam, and it would be more useful if I understood why $I_{Y}(\lambda) \leq I_{X}(\lambda)$ follows from $Y$ being a function of $X$.

EDIT: I have managed to find one mention of this inequality at http://cs.stanford.edu/~ppasupat/a9online/1237.html:

For other statistics $T(X)$, $I_{T}(\theta) \leq I_{X}(\theta)$.

Alas, no proof.

$\endgroup$
  • 2
    $\begingroup$ I'll try fisher information as the names suggest is a way to measure the amount of information X has on the parameter $\lambda$. Since, $Y$ is a function of $X$, $X$ must carry more information than $Y$. My guess that for all function of X that are not Injective the inequality would be sharp, and for function that are injective it would be equal (that is if $y = x + 2$ then $I_y(\lambda) = I_x(\lambda)$). $\endgroup$ – Kozolovska Apr 23 '17 at 7:07
  • $\begingroup$ See Lemma 1.3.2 in stat.tamu.edu/~suhasini/teaching613/chapter1.pdf $\endgroup$ – S. Catterall Reinstate Monica Apr 28 '17 at 14:07
1
$\begingroup$

I am not aware of any general rule to pass over to the Fisher information of the floor function, but in this case it is possible to solve the problem for the specific distribution. The easiest way to do this is to explicitly determine the distribution of $Y = \text{floor} (X)$ and then find the Fisher information for the discrete distribution. For each possible argument value of the floor $y = 0,1,2, ...$ we have:

$$p_Y(y) = \mathbb{P}(y \leqslant X < y+1) = (1-e^{-\lambda(y+1)}) - (1-e^{-\lambda y}) = (1 - e^{-\lambda}) e^{-\lambda y}.$$

This is a discrete distribution, with expected value:

$$\mathbb{E}(Y) = \sum_{y=0}^\infty y \cdot p_Y(y) = (1 - e^{-\lambda}) \sum_{y=0}^\infty y e^{-\lambda y} = \frac{e^{-\lambda}}{1-e^{-\lambda}}.$$

This distribution has score function:

$$s_y(\lambda) \equiv \frac{\partial}{\partial \lambda} \ln p_Y(y | \lambda) = \frac{\partial}{\partial \lambda} ( \ln(1 - e^{-\lambda}) -\lambda y ) = \frac{e^{-\lambda}}{1-e^{-\lambda}} - y.$$

Since the expected score is zero, the Fisher information is given by:

$$\begin{equation} \begin{aligned} I_Y(\lambda) &= \mathbb{E} \Big[ s_Y(\lambda)^2 \Big] \\[6pt] &= \sum_{y=0}^\infty \Big( \frac{e^{-\lambda}}{1-e^{-\lambda}} - y \Big)^2 p_Y(y) \\[6pt] &= \sum_{y=0}^\infty \Big( \frac{e^{-2\lambda}}{(1-e^{-\lambda})^2} - \frac{2e^{-\lambda}}{1-e^{-\lambda}} \cdot y + y^2 \Big) (1 - e^{-\lambda}) e^{-\lambda y} \\[6pt] &= \frac{e^{-2\lambda}}{1-e^{-\lambda}} \sum_{y=0}^\infty e^{-\lambda y} - 2 e^{-\lambda} \sum_{y=0}^\infty y e^{-\lambda y} + (1 - e^{-\lambda}) \sum_{y=0}^\infty y^2 e^{-\lambda y} \\[6pt] &= \frac{e^{-2\lambda}}{(1-e^{-\lambda})^2} - \frac{2 e^{-2\lambda}}{(1-e^{-\lambda})^2} + \frac{e^{-\lambda}(1+e^{-\lambda})}{(1-e^{-\lambda})^2} \\[6pt] &= \frac{1}{(1-e^{-\lambda})^2} \Bigg[e^{-2\lambda} - 2 e^{-2\lambda} + e^{-\lambda}(1+e^{-\lambda}) \Bigg] \\[6pt] &= \frac{1}{(1-e^{-\lambda})^2} \Bigg[e^{-2\lambda} - 2 e^{-2\lambda} + e^{-\lambda}+e^{-2\lambda} \Bigg] \\[6pt] &= \frac{e^{-\lambda}}{(1-e^{-\lambda})^2}. \\[6pt] \end{aligned} \end{equation}$$

We can now verify the inequality $I_Y(\lambda) \leqslant I_X(\lambda)$ directly. Using a bit of algebra, and expanding the exponentials using the Maclaurin series representation, we obtain:

$$\begin{equation} \begin{aligned} I_Y(\lambda) = \frac{e^{-\lambda}}{(1-e^{-\lambda})^2} &= \frac{e^{-\lambda}}{1 - 2e^{-\lambda} + e^{-2\lambda}} \\[6pt] &= \Big( \frac{1 - 2e^{-\lambda} + e^{-2\lambda}}{e^{-\lambda}} \Big)^{-1}\\[6pt] &= (e^{\lambda} - 2 + e^{-\lambda})^{-1} \\[6pt] &= \Big( \sum_{k=0}^\infty \frac{\lambda^k}{k!} - 2 + \sum_{k=0}^\infty \frac{(-1)^k \lambda^k}{k!} \Big)^{-1} \\[6pt] &= \Big( 1 + \sum_{k=1}^\infty \frac{\lambda^k}{k!} - 2 + 1 + \sum_{k=1}^\infty \frac{(-1)^k \lambda^k}{k!} \Big)^{-1} \\[6pt] &= \Big( \sum_{k=1}^\infty \frac{\lambda^k}{k!} + \sum_{k=1}^\infty \frac{(-1)^k \lambda^k}{k!} \Big)^{-1} \\[6pt] &= \Big( 2 \sum_{k=1}^\infty \frac{\lambda^{2k}}{(2k)!} \Big)^{-1} \\[6pt] &= \Big( \sum_{k=1}^\infty \frac{\lambda^{2k}}{(2k)! / 2} \Big)^{-1} \\[6pt] &= \frac{1}{ \lambda^2 + \frac{\lambda^4}{12} + \frac{\lambda^6}{360} + \cdots} \\[6pt] &\leqslant \frac{1}{\lambda^2} = I_X(\lambda). \end{aligned} \end{equation}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.