Questions tagged [fisher-information]
The Fisher information measures the curvature of the log-likelihood and can be used to assess the efficiency of estimators.
209
questions
0
votes
0answers
11 views
Does the fisher information matrix exist when we can only calculate the quasi-likelihood and not the full-likelihood function?
Does the fisher information matrix exist when we can only calculate the quasi-likelihood and not the full-likelihood function?
In GEE, the full-likelihood isn't calculated, but the variances are ...
0
votes
2answers
18 views
expected value of a score function (the gradient of the log-likelihood function)
according to the Wikipedia: https://en.wikipedia.org/wiki/Score_(statistics), expected value of a score function should equals to zero and the proof is following:
\begin{equation}
\begin{aligned}
\...
0
votes
1answer
26 views
Clarifying the definition of Fisher information
I'm studying the Fisher information, which leads me to the wiki page:
https://en.wikipedia.org/wiki/Fisher_information
It says:
Formally, the partial derivative with respect to θ of the natural ...
7
votes
1answer
121 views
What is the geometric relationship between the covariance matrix and the inverse of the covariance matrix?
The covariance matrix represents the dispersion of data points while the inverse of the covariance matrix represents the tightness of data points. How is the dispersion and tightness related ...
0
votes
0answers
20 views
MLEs multivariate normal distribution estimation
I’m a beginner in this field, I hope the problem will be clear… .
Under some regularity assumption the MLE estimators of unknown parameters are unbiased and their distributions is a multivariate ...
0
votes
0answers
32 views
Proof Sample Variance is Minimum Variance Unbiased Estimator for Unknown Mean
I am trying to prove that the unbiased sample variance is a minimum variance estimator. In this problem I have a Normal distribution with unknown mean (and the variance is the parameter to estimate so ...
1
vote
1answer
54 views
Cramér-Rao lower bound with an uncertain observable
Let's say I have an observable $X$ that depends on a parameter $\theta$, and that I can find an expression for the Cramér-Rao lower bound for estimating $\theta$, as a function of $X$: $\sigma_{CRB}(X)...
-1
votes
1answer
44 views
Why Fisher Information uses Log Likelihood and not Plain Likelihood [duplicate]
I would like to know that to determine Fisher information from the Likelihood model, why do we take the log of the likelihood first instead of using normal likelihood ?
3
votes
2answers
95 views
Conflicting Definition of Information in Statistics | Fisher Vs Shannon
The notion of information as per Shannon is that if the probability of RV is close to 1, there is little information in that RV because we are more certain about the outcome of the RV so there is ...
1
vote
0answers
53 views
Deriving C-R inequality from H-C-R bound
As mentioned in the title, I want to derive the Cramer-Rao Lower bound from the Hammersly-Chapman-Robbins lower bound for the variance of a statistic $T$.
The statement for the H-C-R lower bound is ...
1
vote
0answers
28 views
Why is the Fisher information of a one-parameter family of exponential distributions the inverse of the variance?
Let
$$
f(x; \lambda) = h(x) e^{\eta(\lambda) J(x) + \xi(\lambda)}
$$
be a one-parameter family of distributions. In my course notes, it is claimed that
$$
I(\lambda) := \mathbb{E}_\lambda\left[ \left(...
1
vote
0answers
25 views
Fisher Information | Second Derivative of Likelihood Vs Second Derivative of Log Likelihood
I watched this video on Fisher Information and it is mentioned that in Taylor series expansion of the likelihood function the second derivative is parabola which is not a good approximation and a ...
1
vote
1answer
68 views
How to compute variance of Cox model coefficient estimate using Fisher information?
We have Cox proportional hazards model:
$$ \lambda(t,x) = \lambda_0(t)exp(\boldsymbol \beta'\boldsymbol x),$$
where $\boldsymbol \beta$ and $\boldsymbol x$ are vectors.
To make it simple, lets say ...
3
votes
0answers
44 views
Inequalities on Fisher Information / expected second derivative?
Under some regularity conditions we can compute fisher information as
$ - \mathbb{E}_{\theta_0} [\frac{\partial}{\partial \theta^2} \ln f(x;\theta_0)] $
I was wondering if there are some kind of ...
1
vote
0answers
63 views
Fisher's information for weibull 2 parameter
I want to find Fisher's information $I(\theta)$ with $-E(\frac{d^2(ln(fx))}{d\theta^2})$ but in two parameters, the expectation is too complex for me. Is there something about the Weibull distribution ...
1
vote
0answers
19 views
Connection between the two expressions of fisher information
I've seen two major expressions for the fisher information:
$$I_{\mathbf{X}}(\theta) = \mathbb{V}ar(\ell'(\theta|\mathbf{X}))$$
where $\ell(\theta)$ is the log-likelihood function for $\mathbf{X}$...
7
votes
2answers
367 views
Differences between Sampler, MonteCarlo, Metropolis-Hasting method, MCMC method and Fisher formalism
1) I make confusions about what we call a "sampler". From what I understand, a sampler allows to generate a distribution of points that follows a known PDF (probability distribution function), doesn't ...
1
vote
0answers
135 views
MLE for bivariate Normal distribution and Fisher Information
I got very confused trying to understand the meaning of Fisher information, and link it with the information for parameters contained in samples.
Suppose $(X_1,X_2) \sim \mathcal{N}(\begin{bmatrix} \...
0
votes
0answers
14 views
What is the difference between standard errors using the inverse of hessian and calculated using the the inverse of hessian and Fisher information?
In one of R packages for advanced survival analysis, the frailtypack, the output contains standard errors calculated in two ways, named: H (using the inverse of Hessian) and HIH (...
0
votes
0answers
16 views
Score asymptotically belongs to the span of Fisher's information matrix
I am reading a paper from Poskitt and Tremayne titled "Testing the specification of a fitted autoregressive-moving average model".
The paper is concerned with a solution to the singularity of Fisher'...
3
votes
1answer
114 views
explanation of why an UMVUE doesn't necessarily have to achieve the CRLB?
I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have ...
0
votes
0answers
18 views
Information contained in sample about the…estimator?
I have always understood Fisher information to represent the information contained in data regarding some unknown parameter. In my professor's notes, he writes about the information contained in the ...
0
votes
1answer
28 views
Fisher Information for Cox Model
Actually, I'm working on a Statistical Genetics Article (Schaid and al,2010) in a retrospective likelihood context.
In the article, authors present some result about conditional likelihood but I can't ...
1
vote
0answers
26 views
Fisher matrix with penalty function
I am fitting a parametric model to human tracking data. Because my data is in part corrupted (see below why), as such I had to introduce a penalty function to my optimization algorithm.
The Fisher ...
0
votes
0answers
29 views
Different formulas for the Fisher Information, for use in Cramér-Rao lower bound. Correct? Assumptions?
I am taking a course in Statistical Inference. We have to calculate the Cramér-Rao lower bound to determine if different statistics are efficient or not.
I have read that as long as we have an ...
0
votes
0answers
37 views
Fisher Information with respect to the Standard deviation of Normal distribution
Let $X\sim\mathcal{N}(0,\sigma^2)$ be given. I computed the Fisher Information to be $I(\sigma)=\frac{2}{\sigma^2}$. Note that the Fisher Information for the variance is given by $I(\sigma^2)=\frac{1}{...
3
votes
1answer
69 views
Is it true that Fisher information for a statistic and the sample are equal if and only if the statistic is sufficient?
According to https://en.wikipedia.org/wiki/Fisher_information#Sufficient_statistic we have if and only if, but according to https://projecteuclid.org/download/pdfview_1/euclid.imsc/1362751193 we don'...
1
vote
0answers
23 views
How can I find CRLB for an integer valued Maximum Likelihood Estimator?
Reference: How to find Maximum Likelihood estimates of an *integer* parameter?
Extending the above question, how to find the Fisher information and hence CRLB for the integer-valued Estimator using ...
0
votes
0answers
36 views
Fisher Information Matrix for regression problem
In DeepMind's paper [Overcoming catastrophic forgetting in neural networks](
https://arxiv.org/abs/1612.00796), elastic weight consolidation with Fisher Information Matrix is used to tackle the ...
2
votes
0answers
29 views
Signal-to-noise-ratio, Fisher information and and “estimability”
Given a parametric statistical model, is it common to study the quantity
$$ Q_{\theta} = \theta^2 I_{\theta} \, ,$$
where $I_{\theta}$ is the Fisher information? (I focus on a single parameter for ...
1
vote
1answer
99 views
Fisher Information for general one parameter exponential family (inconsistent with Poisson distribution)
For one of my hw questions, I was asked to derive Fisher Information for one parameter exponential family.
Here's my approach:
$$L(\theta) = f(x\mid\theta) = e^{\theta T(x) - \eta(\theta)}h(x)$$
$$\...
0
votes
0answers
20 views
Sampling distribution of the Score statistics of a GLM MODEL
In the context of a GLM (with a distribution that belongs to the exponential family), we often compute the score statistics
$$
U = \frac{\partial LogLike(\boldsymbol{\beta};\mathbf{y})}{\partial\...
0
votes
0answers
173 views
Fisher Information for Gamma
Question: Find the fisher's information for $\mathcal{G}(\alpha, \beta)$ ,$\beta$ known.
Attempt: Since $\mathcal{G}(\alpha, \beta)$ ,$\beta$ known.
$$f(x|\alpha) =\dfrac{x^{\alpha-1}e^{x/\beta}}...
0
votes
0answers
30 views
Fisher information for double exponential distribution [duplicate]
I am asked to find the Fisher information function for double exponential distribution with mean $\mu$ and variance 1.
Here's my approach:
$l(\mu) = -\frac{1}{2}e^{|x-\mu|}$,
so $\log l(\mu) = ...
1
vote
0answers
40 views
Compute the information matries related to normal distribution
This is a problem that I have trouble with.
Suppose that we have $X_{1}, \ldots, X_{m}$ are iid
$N\left(\mu, \sigma^{2}\right), Y_{1}, \ldots, Y_{n}$ are iid $N\left(0, \sigma^{2}\right),$ the $X$...
0
votes
0answers
40 views
Why do we use $S^2$ while estimating the variance?
Sorry the title is a bit silly, but I currently confront a problem related to Fisher's information.
Let $X_1, X_2, \cdots, X_n$ be of $N(\mu , \sigma^2 )$ distribution where $\mu$ is known, $U^2 := ...
0
votes
1answer
548 views
Fisher information from MLE in R?
Reworded the question:
I have read "The Fisher information I(p) is this negative second derivative of the log-likelihood function, averaged over all possible X = {h, N–h}, when we assume some value ...
0
votes
0answers
148 views
Inverse Covariance Matrix of a Gaussian Distribution: Relationship of Precision Matrix and Information Matrix
In the book "Probabilistic Robotics" (Thrun et al.), chapter 3.5.1 states that
The canonical parameterization of a multivariate Gaussian is given by a matrix $\Omega$ and a vector $\xi$. The ...
0
votes
1answer
189 views
Calculate the uncertainty of a MLE
I have minimized the negative LL of a Poisson distribution to get an MLE of three parameters using scipy.minimize w/ Nelder-Mead. I want to calculate the uncertainty of the MLE.
From reading, I ...
0
votes
0answers
16 views
Does C-efficiency exist? If so, how is it defined?
When using an algorithm to construct an optimal experimental design one has a measurement of optimality to find an optimal design with this optimality-criterion. For any arbitrary chosen design a D- ...
0
votes
0answers
67 views
Asymptotic Mean Squared Error of Maximum Likelihood estimator
I want to show that $n$ times mean squared error for the maximum likelihood estimator converges to the inverse of Fisher information, where $n$ is the number of samples.
But The standard proofs of ...
1
vote
1answer
56 views
Statistical comparison of (covariance) matrices
I am trying to test whether the covariance matrix for the maximum likelihood estimates for a gaussian general lienear model approaches the inverse Fisher information matrix (times 1/n , n being the ...
0
votes
0answers
15 views
Does Fisher Information quantify precision? [duplicate]
Looking at perspective from estimating the actual value from a set of data measured by the instrument. Does Fisher information just quantify the precision of the measurement?
What does it say about ...
0
votes
0answers
40 views
What statistic to use in testing the variance of maximum likelihood estimators
(A physicist self-studying statistics here)
I was previously confused about the meaning of the standard error in a maximum likelihood estimate. Certain stack exchange posts (linked below) have gone ...
0
votes
0answers
22 views
Efficient GPU (batch) implementation of Empirical Fisher information matrix?
I have seen many implementations. It seems to be a limitation of autograd itself that we can compute the gradient of loglikelihood only one sample at a time.
The batch version has been used but in a ...
0
votes
0answers
14 views
Action of Fisher information in “Latent Variable Model Selection…”
I'm having trouble understanding the role of the Fisher information matrix in the assumptions of Chandrasekaran et al. 2012.
In the paper, the authors define the Fisher information matrix (i.e., ...
0
votes
1answer
45 views
Sufficient Statistic and Unbiased Estimate in Exponential Family
I am reading this classic paper (Information and the Accuracy Attainable in the Estimation of Statistical Parameters) by CR Rao where he deals with sufficient statistics in exponential distributions ...
3
votes
2answers
620 views
Score function of poisson distribution
When counting the score for poisson distribution I get the log likelihood
$$S(\mu ) = \frac{\partial \ell(\lambda )}{\partial \lambda } = \sum_1^n \left(\frac{y_i}{\lambda}-1\right)$$
Textbook says ...
3
votes
0answers
43 views
Fisher information matrix and gradients
I'm a math Ph.D. without formal training in statistics. Quite a few papers on normalization methods in deep learning mention the Fisher information matrix and how it's related to the Riemannian metric ...
2
votes
1answer
41 views
How do we deduce this fisher information relation?
Given a RS $X_{1},X_{2},\ldots,X_{n}$ whose distribution is well known (unless its parameters), how do we prove the following Fischer Information relationship
\begin{align*}
I_{F}(\theta) =\textbf{E}\...
