Linked Questions

0 votes
0 answers
187 views

Interpretation of the elements of the error matrix as inverse of hessian matrix [duplicate]

In a report I am reading at work, the error matrix is calculated as the inverse of the hessian matrix and used to calculate the error ellipse angle and axes with a not theoretically correct formula. ...
cicciodevoto's user avatar
1 vote
0 answers
30 views

How do I construct a variance-covariance matrix from a matrix formulation of a MLR? [duplicate]

I'm trying to calculate the SE for each coefficient given by a matrix formulation of MLR by the root of each diagonal in the so-called variance-covariance matrix, but I'm unsure how to construct this ...
Jackson Capper's user avatar
22 votes
1 answer
19k views

Relationship between Hessian Matrix and Covariance Matrix

While I am studying Maximum Likelihood Estimation, to do inference in Maximum Likelihood Estimaion, we need to know the variance. To find out the variance, I need to know the Cramer's Rao Lower Bound, ...
user122358's user avatar
  • 1,673
12 votes
1 answer
9k views

Why is the Fisher information the inverse of the (asymptotic) covariance, and vice versa?

For the multinomial distribution, I had spent a lot of time and effort calculating the inverse of the Fisher information (for a single trial) using things like the Sherman-Morrison formula. But ...
Chill2Macht's user avatar
  • 6,169
11 votes
1 answer
8k views

How to estimate confidence interval of a least-squares fit parameters by means of numerical Jacobian

I am working on a complicated data fitting algorithm in Matlab. I have a problem with properly estimating the confidence intervals of my fit. I will describe my procedure in some detail, give some of ...
MarcinKonowalczyk's user avatar
4 votes
2 answers
5k views

How to compute (or numerically estimate) the standard error of the MLE

I have a model for which I know the log likelihood function, the gradient of the log likelihood and the Hessian of the log likelihood. For given data I can compute the MLE using a generic optimizer (...
Simd's user avatar
  • 2,019
3 votes
1 answer
4k views

Fisher information from MLE in R?

Reworded the question: I have read "The Fisher information I(p) is this negative second derivative of the log-likelihood function, averaged over all possible X = {h, N–h}, when we assume some value ...
user2720661's user avatar
7 votes
1 answer
3k views

Why need wald test ( a squared version of t test ) when we already have t test?

It seems to me that they are basically calculate the same thing. Since we already have t-test, why do we need a squared version (wald test)? Does wald test have its own advantage? For example in Cox ...
unicorn's user avatar
  • 859
4 votes
1 answer
2k views

Wald test standard error

I want to compute from scratch a Wald test to test significance of one coefficient in a logistic regression model. I've been to so many posts and blogs but couldn't find a clear explanation with ...
aerijman's user avatar
  • 155
3 votes
1 answer
2k views

How does coxph calculate standard errors?

How are the standard errors in the coxph function in R calculated? According to Fox, "The column marked z in the output records the ratio of each regression coefficient to its standard error, a Wald ...
spindoctor's user avatar
5 votes
0 answers
2k views

Expected and observed Fisher information? [duplicate]

Studying asymptotics, I bumped into the concept of Observed Fisher Information, as a way to compute Fisher Information when the parameter $\theta$ is unknown. I am also aware that it is related in ...
PhDing's user avatar
  • 2,999
4 votes
0 answers
2k views

Negative Hessian matrix in R optim [closed]

I used the optim() function in R to find the min log likelihood, however the diagonal elements of the inverse of Hessian matrix turned out to be negative. ...
lsheng's user avatar
  • 141
1 vote
2 answers
350 views

Fisher information vs Posterior Covariance

I have a parameter $\theta$ and data $y = f(\theta) + \mathrm{noise}$. My goal is finding the best fit for $\theta$ and assess the uncertainty I have on this best fit. I see two competing approaches ...
G. Gare's user avatar
  • 73
1 vote
1 answer
701 views

Observed Fisher Information and confidence intervals

I'm trying to put confidence intervals on parameters fitted through MLE through the inversion of the observed Fisher information matrix. More specifically, I define the observed FIM as: $$ J_{n}(\hat{...
auf-wiedersehen-yall's user avatar
1 vote
0 answers
1k views

Finding standard errors of maximum likelihood estimates

Suppose we use Maximum Likelihood estimation to estimate certain parameters in a model. Furthermore, suppose that the log likelihood function can not be solved analytically and thus must be optimised ...
Whizkid95's user avatar
  • 292

15 30 50 per page