# Questions tagged [fisher-information]

The Fisher information measures the curvature of the log-likelihood and can be used to assess the efficiency of estimators.

209 questions
Filter by
Sorted by
Tagged with
11 views

### Does the fisher information matrix exist when we can only calculate the quasi-likelihood and not the full-likelihood function?

Does the fisher information matrix exist when we can only calculate the quasi-likelihood and not the full-likelihood function? In GEE, the full-likelihood isn't calculated, but the variances are ...
18 views

### expected value of a score function (the gradient of the log-likelihood function)

according to the Wikipedia: https://en.wikipedia.org/wiki/Score_(statistics), expected value of a score function should equals to zero and the proof is following: \begin{equation} \begin{aligned} \...
26 views

### Clarifying the definition of Fisher information

I'm studying the Fisher information, which leads me to the wiki page: https://en.wikipedia.org/wiki/Fisher_information It says: Formally, the partial derivative with respect to θ of the natural ...
121 views

### What is the geometric relationship between the covariance matrix and the inverse of the covariance matrix?

The covariance matrix represents the dispersion of data points while the inverse of the covariance matrix represents the tightness of data points. How is the dispersion and tightness related ...
20 views

### MLEs multivariate normal distribution estimation

I’m a beginner in this field, I hope the problem will be clear… . Under some regularity assumption the MLE estimators of unknown parameters are unbiased and their distributions is a multivariate ...
32 views

### Proof Sample Variance is Minimum Variance Unbiased Estimator for Unknown Mean

I am trying to prove that the unbiased sample variance is a minimum variance estimator. In this problem I have a Normal distribution with unknown mean (and the variance is the parameter to estimate so ...
54 views

Let's say I have an observable $X$ that depends on a parameter $\theta$, and that I can find an expression for the Cramér-Rao lower bound for estimating $\theta$, as a function of $X$: $\sigma_{CRB}(X)... 1answer 44 views ### Why Fisher Information uses Log Likelihood and not Plain Likelihood [duplicate] I would like to know that to determine Fisher information from the Likelihood model, why do we take the log of the likelihood first instead of using normal likelihood ? 2answers 95 views ### Conflicting Definition of Information in Statistics | Fisher Vs Shannon The notion of information as per Shannon is that if the probability of RV is close to 1, there is little information in that RV because we are more certain about the outcome of the RV so there is ... 0answers 53 views ### Deriving C-R inequality from H-C-R bound As mentioned in the title, I want to derive the Cramer-Rao Lower bound from the Hammersly-Chapman-Robbins lower bound for the variance of a statistic$T$. The statement for the H-C-R lower bound is ... 0answers 28 views ### Why is the Fisher information of a one-parameter family of exponential distributions the inverse of the variance? Let $$f(x; \lambda) = h(x) e^{\eta(\lambda) J(x) + \xi(\lambda)}$$ be a one-parameter family of distributions. In my course notes, it is claimed that $$I(\lambda) := \mathbb{E}_\lambda\left[ \left(... 0answers 25 views ### Fisher Information | Second Derivative of Likelihood Vs Second Derivative of Log Likelihood I watched this video on Fisher Information and it is mentioned that in Taylor series expansion of the likelihood function the second derivative is parabola which is not a good approximation and a ... 1answer 68 views ### How to compute variance of Cox model coefficient estimate using Fisher information? We have Cox proportional hazards model:$$ \lambda(t,x) = \lambda_0(t)exp(\boldsymbol \beta'\boldsymbol x),$$where \boldsymbol \beta and \boldsymbol x are vectors. To make it simple, lets say ... 0answers 44 views ### Inequalities on Fisher Information / expected second derivative? Under some regularity conditions we can compute fisher information as - \mathbb{E}_{\theta_0} [\frac{\partial}{\partial \theta^2} \ln f(x;\theta_0)] I was wondering if there are some kind of ... 0answers 63 views ### Fisher's information for weibull 2 parameter I want to find Fisher's information I(\theta) with -E(\frac{d^2(ln(fx))}{d\theta^2}) but in two parameters, the expectation is too complex for me. Is there something about the Weibull distribution ... 0answers 19 views ### Connection between the two expressions of fisher information I've seen two major expressions for the fisher information:$$I_{\mathbf{X}}(\theta) = \mathbb{V}ar(\ell'(\theta|\mathbf{X}))$$where \ell(\theta) is the log-likelihood function for \mathbf{X}... 2answers 367 views ### Differences between Sampler, MonteCarlo, Metropolis-Hasting method, MCMC method and Fisher formalism 1) I make confusions about what we call a "sampler". From what I understand, a sampler allows to generate a distribution of points that follows a known PDF (probability distribution function), doesn't ... 0answers 135 views ### MLE for bivariate Normal distribution and Fisher Information I got very confused trying to understand the meaning of Fisher information, and link it with the information for parameters contained in samples. Suppose (X_1,X_2) \sim \mathcal{N}(\begin{bmatrix} \... 0answers 14 views ### What is the difference between standard errors using the inverse of hessian and calculated using the the inverse of hessian and Fisher information? In one of R packages for advanced survival analysis, the frailtypack, the output contains standard errors calculated in two ways, named: H (using the inverse of Hessian) and HIH (... 0answers 16 views ### Score asymptotically belongs to the span of Fisher's information matrix I am reading a paper from Poskitt and Tremayne titled "Testing the specification of a fitted autoregressive-moving average model". The paper is concerned with a solution to the singularity of Fisher'... 1answer 114 views ### explanation of why an UMVUE doesn't necessarily have to achieve the CRLB? I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have ... 0answers 18 views ### Information contained in sample about the…estimator? I have always understood Fisher information to represent the information contained in data regarding some unknown parameter. In my professor's notes, he writes about the information contained in the ... 1answer 28 views ### Fisher Information for Cox Model Actually, I'm working on a Statistical Genetics Article (Schaid and al,2010) in a retrospective likelihood context. In the article, authors present some result about conditional likelihood but I can't ... 0answers 26 views ### Fisher matrix with penalty function I am fitting a parametric model to human tracking data. Because my data is in part corrupted (see below why), as such I had to introduce a penalty function to my optimization algorithm. The Fisher ... 0answers 29 views ### Different formulas for the Fisher Information, for use in Cramér-Rao lower bound. Correct? Assumptions? I am taking a course in Statistical Inference. We have to calculate the Cramér-Rao lower bound to determine if different statistics are efficient or not. I have read that as long as we have an ... 0answers 37 views ### Fisher Information with respect to the Standard deviation of Normal distribution Let X\sim\mathcal{N}(0,\sigma^2) be given. I computed the Fisher Information to be I(\sigma)=\frac{2}{\sigma^2}. Note that the Fisher Information for the variance is given by I(\sigma^2)=\frac{1}{... 1answer 69 views ### Is it true that Fisher information for a statistic and the sample are equal if and only if the statistic is sufficient? According to https://en.wikipedia.org/wiki/Fisher_information#Sufficient_statistic we have if and only if, but according to https://projecteuclid.org/download/pdfview_1/euclid.imsc/1362751193 we don'... 0answers 23 views ### How can I find CRLB for an integer valued Maximum Likelihood Estimator? Reference: How to find Maximum Likelihood estimates of an *integer* parameter? Extending the above question, how to find the Fisher information and hence CRLB for the integer-valued Estimator using ... 0answers 36 views ### Fisher Information Matrix for regression problem In DeepMind's paper [Overcoming catastrophic forgetting in neural networks]( https://arxiv.org/abs/1612.00796), elastic weight consolidation with Fisher Information Matrix is used to tackle the ... 0answers 29 views ### Signal-to-noise-ratio, Fisher information and and “estimability” Given a parametric statistical model, is it common to study the quantity$$ Q_{\theta} = \theta^2 I_{\theta} \, ,$$where I_{\theta} is the Fisher information? (I focus on a single parameter for ... 1answer 99 views ### Fisher Information for general one parameter exponential family (inconsistent with Poisson distribution) For one of my hw questions, I was asked to derive Fisher Information for one parameter exponential family. Here's my approach:$$L(\theta) = f(x\mid\theta) = e^{\theta T(x) - \eta(\theta)}h(x)\... 0answers 20 views ### Sampling distribution of the Score statistics of a GLM MODEL In the context of a GLM (with a distribution that belongs to the exponential family), we often compute the score statistics $$U = \frac{\partial LogLike(\boldsymbol{\beta};\mathbf{y})}{\partial\... 0answers 173 views ### Fisher Information for Gamma Question: Find the fisher's information for \mathcal{G}(\alpha, \beta) ,\beta known. Attempt: Since \mathcal{G}(\alpha, \beta) ,\beta known.$$f(x|\alpha) =\dfrac{x^{\alpha-1}e^{x/\beta}}... 0answers 30 views ### Fisher information for double exponential distribution [duplicate] I am asked to find the Fisher information function for double exponential distribution with mean$\mu$and variance 1. Here's my approach:$l(\mu) = -\frac{1}{2}e^{|x-\mu|}$, so$\log l(\mu) = ...
This is a problem that I have trouble with. Suppose that we have $X_{1}, \ldots, X_{m}$ are iid $N\left(\mu, \sigma^{2}\right), Y_{1}, \ldots, Y_{n}$ are iid $N\left(0, \sigma^{2}\right),$ the $X$...