Linked Questions

1
vote
0answers
25 views

How do I construct a variance-covariance matrix from a matrix formulation of a MLR? [duplicate]

I'm trying to calculate the SE for each coefficient given by a matrix formulation of MLR by the root of each diagonal in the so-called variance-covariance matrix, but I'm unsure how to construct this ...
16
votes
1answer
12k views

Relationship between Hessian Matrix and Covariance Matrix

While I am studying Maximum Likelihood Estimation, to do inference in Maximum Likelihood Estimaion, we need to know the variance. To find out the variance, I need to know the Cramer's Rao Lower Bound, ...
8
votes
1answer
5k views

Why is the Fisher information the inverse of the (asymptotic) covariance, and vice versa?

For the multinomial distribution, I had spent a lot of time and effort calculating the inverse of the Fisher information (for a single trial) using things like the Sherman-Morrison formula. But ...
6
votes
1answer
6k views

How to estimate confidence interval of a least-squares fit parameters by means of numerical Jacobian

I am working on a complicated data fitting algorithm in Matlab. I have a problem with properly estimating the confidence intervals of my fit. I will describe my procedure in some detail, give some of ...
3
votes
2answers
4k views

How to compute (or numerically estimate) the standard error of the MLE

I have a model for which I know the log likelihood function, the gradient of the log likelihood and the Hessian of the log likelihood. For given data I can compute the MLE using a generic optimizer (...
1
vote
1answer
2k views

Fisher information from MLE in R?

Reworded the question: I have read "The Fisher information I(p) is this negative second derivative of the log-likelihood function, averaged over all possible X = {h, N–h}, when we assume some value ...
7
votes
1answer
1k views

Why need wald test ( a squared version of t test ) when we already have t test?

It seems to me that they are basically calculate the same thing. Since we already have t-test, why do we need a squared version (wald test)? Does wald test have its own advantage? For example in Cox ...
5
votes
0answers
2k views

Expected and observed Fisher information? [duplicate]

Studying asymptotics, I bumped into the concept of Observed Fisher Information, as a way to compute Fisher Information when the parameter $\theta$ is unknown. I am also aware that it is related in ...
4
votes
0answers
2k views

Negative Hessian matrix in R optim [closed]

I used the optim() function in R to find the min log likelihood, however the diagonal elements of the inverse of Hessian matrix turned out to be negative. ...
1
vote
1answer
1k views

How does coxph calculate standard errors?

How are the standard errors in the coxph function in R calculated? According to Fox, "The column marked z in the output records the ratio of each regression coefficient to its standard error, a Wald ...
1
vote
0answers
1k views

Finding standard errors of maximum likelihood estimates

Suppose we use Maximum Likelihood estimation to estimate certain parameters in a model. Furthermore, suppose that the log likelihood function can not be solved analytically and thus must be optimised ...
3
votes
1answer
656 views

Wald test standard error

I want to compute from scratch a Wald test to test significance of one coefficient in a logistic regression model. I've been to so many posts and blogs but couldn't find a clear explanation with ...
0
votes
0answers
561 views

Bayesian CLT with grid approximation

I am trying do inference on a problem with a posterior distribution with two parameters (not in closed form). I have built a grid approximation and have been able to create a contour plot of this ...
1
vote
1answer
68 views

Observed Fisher Information and confidence intervals

I'm trying to put confidence intervals on parameters fitted through MLE through the inversion of the observed Fisher information matrix. More specifically, I define the observed FIM as: $$ J_{n}(\hat{...
0
votes
0answers
71 views

MLE, standarderrors from Hessian versus OLS standarderrors

I am minimizing a negative log likelihood for OLS using multiple predictors (the X matrix is generated using a D-optimal design).The standard errors should be the square root of the diagional elements ...

15 30 50 per page