Linked Questions

1
vote
0answers
14 views

How do I construct a variance-covariance matrix from a matrix formulation of a MLR? [duplicate]

I'm trying to calculate the SE for each coefficient given by a matrix formulation of MLR by the root of each diagonal in the so-called variance-covariance matrix, but I'm unsure how to construct this ...
10
votes
1answer
6k views

Relationship between Hessian Matrix and Covariance Matrix

While I am studying Maximum Likelihood Estimation, to do inference in Maximum Likelihood Estimaion, we need to know the variance. To find out the variance, I need to know the Cramer's Rao Lower Bound, ...
5
votes
1answer
2k views

Why is the Fisher information the inverse of the (asymptotic) covariance, and vice versa?

For the multinomial distribution, I had spent a lot of time and effort calculating the inverse of the Fisher information (for a single trial) using things like the Sherman-Morrison formula. But ...
6
votes
1answer
3k views

How to estimate confidence interval of a least-squares fit parameters by means of numerical Jacobian

I am working on a complicated data fitting algorithm in Matlab. I have a problem with properly estimating the confidence intervals of my fit. I will describe my procedure in some detail, give some of ...
3
votes
2answers
2k views

How to compute (or numerically estimate) the standard error of the MLE

I have a model for which I know the log likelihood function, the gradient of the log likelihood and the Hessian of the log likelihood. For given data I can compute the MLE using a generic optimizer (...
4
votes
0answers
1k views

Expected and observed Fisher information?

Studying asymptotics, I bumped into the concept of Observed Fisher Information, as a way to compute Fisher Information when the parameter $\theta$ is unknown. I am also aware that it is related in ...
4
votes
0answers
1k views

Negative Hessian matrix in R optim [closed]

I used the optim() function in R to find the min log likelihood, however the diagonal elements of the inverse of Hessian matrix turned out to be negative. ...
0
votes
1answer
515 views

How does coxph calculate standard errors?

How are the standard errors in the coxph function in R calculated? According to Fox, "The column marked z in the output records the ratio of each regression coefficient to its standard error, a Wald ...
0
votes
0answers
463 views

Bayesian CLT with grid approximation

I am trying do inference on a problem with a posterior distribution with two parameters (not in closed form). I have built a grid approximation and have been able to create a contour plot of this ...
1
vote
0answers
282 views

Finding standard errors of maximum likelihood estimates

Suppose we use Maximum Likelihood estimation to estimate certain parameters in a model. Furthermore, suppose that the log likelihood function can not be solved analytically and thus must be optimised ...
0
votes
1answer
169 views

Fisher information from MLE in R?

Reworded the question: I have read "The Fisher information I(p) is this negative second derivative of the log-likelihood function, averaged over all possible X = {h, N–h}, when we assume some value ...
2
votes
1answer
90 views

Why need wald test ( a squared version of t test ) when we already have t test?

It seems to me that they are basically calculate the same thing. Since we already have t-test, why do we need a squared version (wald test)? Does wald test have its own advantage? For example in Cox ...
1
vote
1answer
67 views

Wald test standard error

I want to compute from scratch a Wald test to test significance of one coefficient in a logistic regression model. I've been to so many posts and blogs but couldn't find a clear explanation with ...
0
votes
0answers
49 views

Obtaining Uncertainity from MLE [duplicate]

I read the following statement online: A staple of frequentist statistics is the maximal likelihood estimate. This provides a single number which is often interpreted as being the “most likely” ...