25 views

How do I construct a variance-covariance matrix from a matrix formulation of a MLR? [duplicate]

I'm trying to calculate the SE for each coefficient given by a matrix formulation of MLR by the root of each diagonal in the so-called variance-covariance matrix, but I'm unsure how to construct this ...
12k views

Relationship between Hessian Matrix and Covariance Matrix

While I am studying Maximum Likelihood Estimation, to do inference in Maximum Likelihood Estimaion, we need to know the variance. To find out the variance, I need to know the Cramer's Rao Lower Bound, ...
5k views

Why is the Fisher information the inverse of the (asymptotic) covariance, and vice versa?

For the multinomial distribution, I had spent a lot of time and effort calculating the inverse of the Fisher information (for a single trial) using things like the Sherman-Morrison formula. But ...
6k views

How to estimate confidence interval of a least-squares fit parameters by means of numerical Jacobian

I am working on a complicated data fitting algorithm in Matlab. I have a problem with properly estimating the confidence intervals of my fit. I will describe my procedure in some detail, give some of ...
4k views

How to compute (or numerically estimate) the standard error of the MLE

I have a model for which I know the log likelihood function, the gradient of the log likelihood and the Hessian of the log likelihood. For given data I can compute the MLE using a generic optimizer (...
2k views

Fisher information from MLE in R?

Reworded the question: I have read "The Fisher information I(p) is this negative second derivative of the log-likelihood function, averaged over all possible X = {h, N–h}, when we assume some value ...
1k views

Why need wald test ( a squared version of t test ) when we already have t test?

It seems to me that they are basically calculate the same thing. Since we already have t-test, why do we need a squared version (wald test)? Does wald test have its own advantage? For example in Cox ...
2k views

Expected and observed Fisher information? [duplicate]

Studying asymptotics, I bumped into the concept of Observed Fisher Information, as a way to compute Fisher Information when the parameter $\theta$ is unknown. I am also aware that it is related in ...
2k views

Negative Hessian matrix in R optim [closed]

I used the optim() function in R to find the min log likelihood, however the diagonal elements of the inverse of Hessian matrix turned out to be negative. ...
1k views

How does coxph calculate standard errors?

How are the standard errors in the coxph function in R calculated? According to Fox, "The column marked z in the output records the ratio of each regression coefficient to its standard error, a Wald ...
1k views

Finding standard errors of maximum likelihood estimates

Suppose we use Maximum Likelihood estimation to estimate certain parameters in a model. Furthermore, suppose that the log likelihood function can not be solved analytically and thus must be optimised ...
656 views

Wald test standard error

I want to compute from scratch a Wald test to test significance of one coefficient in a logistic regression model. I've been to so many posts and blogs but couldn't find a clear explanation with ...
561 views

Bayesian CLT with grid approximation

I am trying do inference on a problem with a posterior distribution with two parameters (not in closed form). I have built a grid approximation and have been able to create a contour plot of this ...
I'm trying to put confidence intervals on parameters fitted through MLE through the inversion of the observed Fisher information matrix. More specifically, I define the observed FIM as:  J_{n}(\hat{...