# Questions tagged [fisher-information]

The Fisher information measures the curvature of the log-likelihood and can be used to assess the efficiency of estimators.

321 questions
Filter by
Sorted by
Tagged with
21 views

### Can probabilistic (e.g., Fisher, Shannon) and non-probabilistic (e.g., Hartley, Kolmogorov) information types be jointly useful?

Suppose you draw a random sample from a probability distribution, with the objective of gaining information about a parameter of that distribution. The inferential usefulness of the probabilistic ...
19 views

55 views

### Using Jeffreys prior for Bernoulli distribution to find the prior of a transformation on p

The question goes like this: Use Jeffreys prior for Bernoulli distribution and find the prior for $\eta$ where: $$\eta(p) = \left(\frac{p}{1-p}\right)$$ So $\eta$ here is some kind of a ...
46 views

### What is the Fisher information matrix in the logit model? [duplicate]

Assume that $Y_1,\dots, Y_n$ follows a Binomial distribution with probability of $p(d_i)$. Assume that the pdf of $Y_i$: $$f(p_i,Y_i)=\binom{n_i}{y_i}p_i^{y_i}(1-p_i)^{n_i-y_i}$$ Assume that a model ...
76 views

### Problem with the Fisher information matrix in case of N measurements of two observables

Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model ...
1 vote
61 views

### The distribution of $Y|aY+b$

What is the distribution of $Y|aY+b$? I'm assuming that $a\neq 0$. A related question here discusses the distribution of $X|X$, which is degenerate. My guess is that the distribution of $Y|aY+b$ ...
1 vote
168 views

### Fisher information vs Posterior Covariance

I have a parameter $\theta$ and data $y = f(\theta) + \mathrm{noise}$. My goal is finding the best fit for $\theta$ and assess the uncertainty I have on this best fit. I see two competing approaches ...
1 vote
40 views

### Expectation of Fisher score not equal to 0 when parametrize Categorical distribution differently

Expectation of Fisher score should equal to zero. The prove can be found in many palces, such as wikipedia. But I tried a categorical distribution that is not parameterizatized minimally, the expected ...
21 views

### How to define the relation between the Fisher metric and KL divergence in the following case

Let we have a point cloud data consists of exactly $n$ distinct points in $\mathbb{R^d}$ that each each point clod is of the form $X=\{x_1,...,x_n/ x_i\in \mathbb{R^d},x_{i}\neq x_{j},i\neq j\}$. The ...
1 vote
49 views

I am reading the paper Efficient Computation of the Fisher Information Matrix in the EM Algorithm (Meng & Spall) and am unsure what the difference (if any) there is between the notation $\partial^... 3 votes 3 answers 365 views ### Can the off-diagonal elements of Fisher information matrix be negative? The concept of Fisher information is new to me and as I understand the diagonal elements of the Fisher information matrix (FIM) are proportional to mean square error (to be precise the inverse of FIM).... 1 vote 2 answers 135 views ### Observed Fisher information for the binomial: How is$I(\hat{\theta}) = \frac{n}{\hat{\theta}(1 - \hat{\theta})}$calculated? I am currently studying the textbook In All Likelihood by Yudi Pawitan. Example 2.10 of chapter 2.5 Maximum and curvature of likelihood says the following: Example 2.10: Based on$x$from the ... 0 votes 0 answers 31 views ### What is the role of information matrix in Likelihood estimation? I couldn't grasp what it refers to exactly so I would like to understand how we use it: from MLE, Score Vector is: $$S(\theta;y) = \frac{\partial l(\theta;y) }{\partial \theta}$$$l$comes from the ... 2 votes 0 answers 60 views ### Lost in Fisher information notation I believe I understand the gist of what Fisher Information is, but I want to be rigorous and I am confused by notation. Also I believe I am masquerading mistakes by ... 0 votes 1 answer 71 views ### Asymptotic covariance matrix of an ML estimator and Fisher information Let$(Y_i, X_i)_{1\leq i \leq n}$be i.i.d. such that$Y_i = (Y_i^A, Y_i^B)' \in \mathbb{R}^2$and$X_i = (X_i^A, X_i^B)' \in \mathbb{R}^2. Suppose that $$Y_i^A = X_i^A \beta_A + \epsilon_i^A,$$ ... 0 votes 1 answer 37 views ### Question about Fisher information matrix on a statistical model Consider a family S of probability density functions on X which is defined as p:X\to\mathbb{R} such that p(x\geq 0) and \int_{X}p(x)dx=1.Suppose each element of S may be parameterized ... 0 votes 0 answers 16 views ### Goodness-of-fit and confidence intervals I am having trouble grasping the subtleties of determining confidence intervals on parameters from degrading the goodness-of-fit (ie shifting \chi^2 or 2\ln {\cal L} by 1). For instance, around ... 1 vote 1 answer 189 views ### Fisher information for the negative binomial distribution I have the negative binomial distribution and want to find the fisher information: I(\theta) = V[\ell'(\theta)] How do i calculate this? I know that the derivative of the log-likelihood is: \ell'(\... 1 vote 3 answers 231 views ### Deriving the asymptotic distribution using delta method I have the density function: P_Y(y) = \sqrt{\frac{1}{2\pi y^3}} \exp\left(-\frac{(y-\mu)^2}{2\mu^2y}\right) If we define r := \mu^2 what is its asymptotic distribution? The right answer is \sqrt{... 0 votes 0 answers 79 views ### Critical Value for a 2-sided Wald's test Consider the following set of hypotheses: H_0:\theta = 1 H_1:\theta ≠ 1 AFAIK, the Wald's test converges in distribution as follows: \sqrt{nI(\hat{\theta}_ {MLE})}(\hat{\theta}_{MLE}-\theta)\... 0 votes 0 answers 48 views ### Fiding the test statistic, using wald test Given the random sample X_1,...,X_n \sim N(\mu, \sigma^2), I want to perform a Wald test for: \mathrm{H}_\mathrm{0}: \mu = \mathrm{\mu}_\mathrm{0} \mathrm{H}_\mathrm{1}: \mu \neq \mathrm{\mu}_\... 3 votes 2 answers 102 views ### How to prove \mathcal{I}_{1}(\eta) = \mathcal{I}_{1}(\theta)[h'(\eta)]^{2} where \mathcal{I}_{1} is the Fisher information and \theta = h(\eta)? I am trying to apply the following definition of Fisher Information: \begin{align*} \mathcal{I}_{1}(\theta) = \mathbb{E}_{\theta}\left[\left(\frac{\partial}{\partial\theta}\ln f(x_{1}|\theta)\right)^{... 0 votes 0 answers 21 views ### Error in sign when deriving Fisher information matrix for a linear model We have that Y_i ~\sim N(\beta x_i,\sigma^2) for i=1,2, \dots,n, all independent where x_i is a known covariate. We would like to derive the Fisher information matirx. I know that the (1,1) ... 0 votes 0 answers 217 views ### Find the Fisher Information for geometric distribution Given X1,\dotsc,Xn \sim \mathcal{Geo}(p) , and I need to find the MLE and the CI for the MLE. I found the MLE for this distribution, using the maximum likelihood function: L(p;X) = (1-p)^(Xi-1) * p... 0 votes 0 answers 90 views ### Intuitively, how is fisher information different from inverse of the variance of the likelihood? If I observe data X then Fisher information is supposed to tell me how concretely I can say that the inferred (mle) value of parameter produced this data (I am not sure if it is accurate to ... 4 votes 1 answer 98 views ### What do you think of this proof for Fisher information? I want to prove This formula: The score function is basically the derivative of the maximum likelihood's log, so to get the information I make another derivative of that: -E[∂/∂θ s(X;θ)] = -E[∂/∂θ ... 0 votes 0 answers 84 views ### Fisher information for predictions Assume I have a model (linear regression, neural network, etc) in the formg(\theta)$and I assume that my data is generated according to$f(x; g(\theta))$(eg$f$is the pdf of a normal Gaussian ... 1 vote 0 answers 42 views ### Derivation of an over-parametrized Fisher information for categorical distribution, with$n\ge3$Having found interesting this earlier discussion I wonder if there is any specific example of building a Fisher information matrix (FIM) in case$n\ge3$. I tried myself, but every time, I am getting ... 7 votes 4 answers 646 views ### Why the variance of Maximum Likelihood Estimator(MLE) will be less than Cramer-Rao Lower Bound(CRLB)? Consider this example. Suppose we have three events to happen with probability$p_1=p_2=\frac{1}{2}\sin ^2\theta ,p_3=\cos ^2\theta $respectively. And we suppose the true value$\theta _0=\frac{\pi}{...
The formula writes as this: $$I(X) = \left(\frac{\partial Z}{\partial X}\right)^\top I(Z) \left(\frac{\partial Z}{\partial X}\right),$$ in particular, if $Z = AX$, we have $$I(X) = A^\top I(Z) A.$$ ...