Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a random sample $(X_1, X_2,...,X_n)$ and I have an estimator $\bar{X_n}=\sum_{i=1}^{n} X_i$

I need to compute the Fisher information of $\bar{X_n}$. The Fisher information is defined as $-E\left(\frac{d^2}{d\theta^2}logL\right)$, where $L$ is the likelihood function.

My question is: to compute the Fisher information of the estimator (NOT the random sample, but instead a function of the random sample), should we take the likelihood function of the random sample or the likelihood function of the distribution?

share|improve this question
Could you tell us what you mean by a "likelihood function of [a] distribution"? – whuber Jan 16 '14 at 22:57
Well. The likelihood function of the random sample would be the joint distribution of the n random variables. The likelihood function of the estimator would be the probability of observing a specific value of the estimator... – user114618 Jan 16 '14 at 23:34
OK, thanks for the clarification. Consider what the difference in $\frac{d^2}{d\theta^2}\log L$ would be between the two likelihoods. – whuber Jan 16 '14 at 23:40
They are not always the same! Consider $f_{x_i}(x_i,\theta)=exp[-(x_i-\theta)]\cdot I_{(\theta,+\infty)} (x_i)$. The maximum likelihood estimator would be $min(X_i)$ and the two likelihood functions would be $L(X_1,...,X_2)=exp(n\theta-\sum x_i)\cdot \prod I_{(\theta,+\infty)} (x_i)$ and $L(min(X_i))=n \cdot exp(-n\cdot min(X_i)+n \theta)\cdot I_{(0,+\infty)} (min(X_i)-\theta)$ – user114618 Jan 17 '14 at 9:51
Excellent example! That helps me understand what you are asking. – whuber Jan 17 '14 at 15:26

There is no fisher information of the estimator, just the fisher information of a random sample $\theta$.

In Wikipedia, it says:

In mathematical statistics, the Fisher information (sometimes simply called information1) is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter $\theta$ upon which the probability of X depends.

so, it is true that fisher information is a kind of connection between two random variables, instead of some estimator, which is a function of X.

share|improve this answer
$\theta$ is the parameter — not a sample. The Fisher information is available before you have any examples. – Neil G Feb 7 at 23:35

I'm pretty sure that you've got some terminology mixed up. Fisher's Information is a function of the data, just like an estimator such as $\bar{X}_{n}$ that gives you an idea of how much information of the parameter of interest is contained in the sample you've acquired. You can compute Fisher's Information at an estimator (this is usually done because the F.I. depends on the unknown parameter being estimated) and we use the plug-in estimator consisting of the F.I. evaluated at the MLE (typically).

share|improve this answer
I don't think Fisher information is a function of the data – Neil G Feb 7 at 23:31

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.