6
$\begingroup$

I have a random sample $(X_1, X_2,...,X_n)$ and I have an estimator $\bar{X_n}=\sum_{i=1}^{n} X_i$

I need to compute the Fisher information of $\bar{X_n}$. The Fisher information is defined as $-E\left(\frac{d^2}{d\theta^2}logL\right)$, where $L$ is the likelihood function.

My question is: to compute the Fisher information of the estimator (NOT the random sample, but instead a function of the random sample), should we take the likelihood function of the random sample or the likelihood function of the distribution?

$\endgroup$
7
  • 2
    $\begingroup$ Could you tell us what you mean by a "likelihood function of [a] distribution"? $\endgroup$
    – whuber
    Commented Jan 16, 2014 at 22:57
  • 1
    $\begingroup$ Well. The likelihood function of the random sample would be the joint distribution of the n random variables. The likelihood function of the estimator would be the probability of observing a specific value of the estimator... $\endgroup$
    – user114618
    Commented Jan 16, 2014 at 23:34
  • 3
    $\begingroup$ OK, thanks for the clarification. Consider what the difference in $\frac{d^2}{d\theta^2}\log L$ would be between the two likelihoods. $\endgroup$
    – whuber
    Commented Jan 16, 2014 at 23:40
  • 2
    $\begingroup$ They are not always the same! Consider $f_{x_i}(x_i,\theta)=exp[-(x_i-\theta)]\cdot I_{(\theta,+\infty)} (x_i)$. The maximum likelihood estimator would be $min(X_i)$ and the two likelihood functions would be $L(X_1,...,X_2)=exp(n\theta-\sum x_i)\cdot \prod I_{(\theta,+\infty)} (x_i)$ and $L(min(X_i))=n \cdot exp(-n\cdot min(X_i)+n \theta)\cdot I_{(0,+\infty)} (min(X_i)-\theta)$ $\endgroup$
    – user114618
    Commented Jan 17, 2014 at 9:51
  • 1
    $\begingroup$ I noticed that the sample mean is missing the factor 1/n. $\endgroup$ Commented Jan 8, 2017 at 16:50

2 Answers 2

1
$\begingroup$

There is no fisher information of the estimator, just the fisher information of a random sample $\theta$.

In Wikipedia, it says:

In mathematical statistics, the Fisher information (sometimes simply called information1) is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter $\theta$ upon which the probability of X depends.

so, it is true that fisher information is a kind of connection between two random variables, instead of some estimator, which is a function of X.

$\endgroup$
2
  • 1
    $\begingroup$ $\theta$ is the parameter — not a sample. The Fisher information is available before you have any examples. $\endgroup$
    – Neil G
    Commented Feb 7, 2016 at 23:35
  • $\begingroup$ As noted in the Wikipedia article you cite: "The Fisher information is not a function of a particular observation, as the random variable X has been averaged out." (From context, it is clear that by "observation," they mean the random variable computed on the sample, not any single observation in the sample.) $\endgroup$
    – virtuolie
    Commented Nov 3, 2023 at 16:55
0
$\begingroup$

I'm pretty sure that you've got some terminology mixed up. Fisher's Information is a function of the data, just like an estimator such as $\bar{X}_{n}$ that gives you an idea of how much information of the parameter of interest is contained in the sample you've acquired. You can compute Fisher's Information at an estimator (this is usually done because the F.I. depends on the unknown parameter being estimated) and we use the plug-in estimator consisting of the F.I. evaluated at the MLE (typically).

$\endgroup$
2
  • $\begingroup$ I don't think Fisher information is a function of the data $\endgroup$
    – Neil G
    Commented Feb 7, 2016 at 23:31
  • 1
    $\begingroup$ Random variables are integrated out by expectation as we calculate the Fisher information. So it's not a function of the data; it's rather a function of the parameter. $\endgroup$
    – Daeyoung
    Commented Aug 21, 2016 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.