# Questions tagged [hessian]

For on-topic questions involving the Hessian matrix, a square matrix generalizing the second derivative. Please include also a statistical methods tag. For purely mathemathical questions about the Hessian it is better to ask on math.SE at https://math.stackexchange.com/.

114 questions
Filter by
Sorted by
Tagged with
7 views

• 1,217
1 vote
49 views

### Metropolis Hastings Proposal with Gradient and Hessian Information

I need to sample a high-dimensional parameter vector from a distribution where the gradient, the Hessian and the inverse of the Hessian of the log-likelihood are very cheap to compute. Are there any ...
• 700
340 views

### How to calculate covariance matrix in nonlinear least squares

I am fitting a nonlinear model to observations by using least squares to estimated the model parameters. Theoretically, the covariance matrix of the parameters can be estimated by inverting the ...
39 views

### Computational instability when computing covariance in glm or optim

When we estimate parameters with the maximum likelihood method, then we can estimate the standard error with the Hessian. The code below tries to estimate this for the case of the question: How to ...
• 82.1k
938 views

### scipy minimize gives a hess_inv that is completely different from inv(statmodel.approx_hess)

I'm fitting a model with MLE using scipy.minimize (method BFGS). I want to have the hessian to compute its inverse and retrieve the standard error of each parameter....
41 views

### Coefficient standard error for "GLM" not in exponential family

For GLMs in the exponential family, we can obtain the standard errors for the regression coefficients as a function of the diagonal of the fisher information matrix. Does this still hold if the ...
• 157
70 views

### Large values in the inverse of a Hessian

I'm implementing Newton's method for a simple logistic regression model but I keep getting very large values for the inverse of the Hessian matrix. I am using the standard formulas found in books... I'...
1 vote
56 views

• 1
2k views

### How to Determine Gradient and Hessian for Custom Xgboost Functions? [closed]

I'm trying to tackle a regression problem in which I want to predict data that sometimes has extreme values. The current machine learning algorithm I'm using is xgboost, specifically the python ...
• 91
223 views

### How does the approximate Hessian update in LBFGS work?

Looking at the wikipedia page for BFGS... Wikipedia It looks like a rearranging of Newton's method, but I can't really explain why the update to the approximate Hessian would be given by the following ...
• 942
75 views

### What can a p-value (& sign) tell me about the marginal posterior distribution of a model parameter, and when?

EDIT: The tl;dr here would broadly be: given that both frequentist standard errors and a quadratic approximation of a Bayesian joint posterior can be obtained from the square root of the diagonal ...
378 views

• 942
71 views

### Is it possible to estimate the Hessian as the covariance of primal and cotangent?

Let's say we have a function $$f: \mathbb R^n \to \mathbb R.$$ Can we numerically approximate the Hessian $f''(x)$ as $$\textrm{Var}(a)^{-1} \textrm{Cov}(a, f'(a))$$ where $$E(a) = x?$$
• 15.4k
5k views

### Derivation of Hessian for multinomial logistic regression in Böhning (1992)

This question is basically about row/column notation of derivatives and some basic rules. However, I couldn't figure out where I'm wrong. For multinomial logistic regression, I'm trying to get the ...
• 503
430 views

### Information Matrix for Conditional Likelihood

I am studying the MLE theory on my own and I am confused by the difference between the fisher information matrix for the full sample and for one observation, when it comes to conditional likelihood. ...
• 1,389
128 views

### Interpolation of Hessian matrix

I have a model where hessian matrices are calculated along a path. Since the calculation is done using finite differences, this is very time consuming. I have tried to calculate only every second ...
118 views

• 121
1k views

### Hessian of Ridge Regression

I have a ridge regression problem $$f(W) = \frac{1}{2} ||XW - Y||_2^2 + \lambda W^TW$$ I want to find the smoothness parameter of the function (the $L$ such that $f$ is $L$-smooth). For quadratic ...
• 135
1 vote