This is mainly a reference request. There must be some generalizations of the concept of Fisher information for discrete (say, integer-valued) parameters, and of related results such as the Cramer-Rao bound (or information inequality). I have just never seen them.

Are there any good references, to the concept(s) itself, or to interesting applications?

  • 2
    $\begingroup$ This paper on cardinality estimation uses the Cramér-Rao inequality in a context where the parameter is integer-valued (page 9, proposition 2.9). However, they seem to do some kind of transformation to transform their estimator $\hat \xi$ into $\xi^*$ to make it differentiable. I'm not sure I understand what they're doing exactly, and an answer to your question would probably help… $\endgroup$
    – Ted
    Commented Jun 16, 2017 at 16:01
  • 1
    $\begingroup$ The generalization is the Hammersley–Chapman–Robbins bound, but this bound seems to be too tight for any estimator: the variance decreases exponentially. See my question for an example. $\endgroup$
    – Krivoi
    Commented Jun 13, 2020 at 12:19
  • $\begingroup$ But why in this arxiv, page 5, eq.(1.6), the discrete Fisher information is directly defined? Is it wrong? Cause it seems okay to me. $\endgroup$
    – narip
    Commented Sep 27, 2021 at 2:07
  • 1
    $\begingroup$ @narip: I will look into this again, but note that I ask for a discrete (for example, integer) parameter, not variable. Maybe the misunderstanding is that in some fields the word parameter is used for variable? So, if the parameter is known to be an integer (for example a population size), then the differentiationnused in the usual definition makes no sense! $\endgroup$ Commented Sep 27, 2021 at 2:37
  • 1
    $\begingroup$ @user: NO, I have thought about other things for a while ... $\endgroup$ Commented Nov 24, 2022 at 23:39


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.