5
$\begingroup$

I have a random sample $(X_1, X_2,...,X_n)$ and I have an estimator $\bar{X_n}=\sum_{i=1}^{n} X_i$

I need to compute the Fisher information of $\bar{X_n}$. The Fisher information is defined as $-E\left(\frac{d^2}{d\theta^2}logL\right)$, where $L$ is the likelihood function.

My question is: to compute the Fisher information of the estimator (NOT the random sample, but instead a function of the random sample), should we take the likelihood function of the random sample or the likelihood function of the distribution?

$\endgroup$
  • 2
    $\begingroup$ Could you tell us what you mean by a "likelihood function of [a] distribution"? $\endgroup$ – whuber Jan 16 '14 at 22:57
  • 1
    $\begingroup$ Well. The likelihood function of the random sample would be the joint distribution of the n random variables. The likelihood function of the estimator would be the probability of observing a specific value of the estimator... $\endgroup$ – user114618 Jan 16 '14 at 23:34
  • 3
    $\begingroup$ OK, thanks for the clarification. Consider what the difference in $\frac{d^2}{d\theta^2}\log L$ would be between the two likelihoods. $\endgroup$ – whuber Jan 16 '14 at 23:40
  • 2
    $\begingroup$ They are not always the same! Consider $f_{x_i}(x_i,\theta)=exp[-(x_i-\theta)]\cdot I_{(\theta,+\infty)} (x_i)$. The maximum likelihood estimator would be $min(X_i)$ and the two likelihood functions would be $L(X_1,...,X_2)=exp(n\theta-\sum x_i)\cdot \prod I_{(\theta,+\infty)} (x_i)$ and $L(min(X_i))=n \cdot exp(-n\cdot min(X_i)+n \theta)\cdot I_{(0,+\infty)} (min(X_i)-\theta)$ $\endgroup$ – user114618 Jan 17 '14 at 9:51
  • 1
    $\begingroup$ I noticed that the sample mean is missing the factor 1/n. $\endgroup$ – Michael R. Chernick Jan 8 '17 at 16:50
2
$\begingroup$

There is no fisher information of the estimator, just the fisher information of a random sample $\theta$.

In Wikipedia, it says:

In mathematical statistics, the Fisher information (sometimes simply called information1) is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter $\theta$ upon which the probability of X depends.

so, it is true that fisher information is a kind of connection between two random variables, instead of some estimator, which is a function of X.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ $\theta$ is the parameter — not a sample. The Fisher information is available before you have any examples. $\endgroup$ – Neil G Feb 7 '16 at 23:35
0
$\begingroup$

I'm pretty sure that you've got some terminology mixed up. Fisher's Information is a function of the data, just like an estimator such as $\bar{X}_{n}$ that gives you an idea of how much information of the parameter of interest is contained in the sample you've acquired. You can compute Fisher's Information at an estimator (this is usually done because the F.I. depends on the unknown parameter being estimated) and we use the plug-in estimator consisting of the F.I. evaluated at the MLE (typically).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I don't think Fisher information is a function of the data $\endgroup$ – Neil G Feb 7 '16 at 23:31
  • 1
    $\begingroup$ Random variables are integrated out by expectation as we calculate the Fisher information. So it's not a function of the data; it's rather a function of the parameter. $\endgroup$ – Daeyoung Lim Aug 21 '16 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.