# Questions tagged [hessian]

For on-topic questions involving the Hessian matrix, a square matrix generalizing the second derivative. Please include also a statistical methods tag. For purely mathemathical questions about the Hessian it is better to ask on math.SE at https://math.stackexchange.com/.

63 questions
Filter by
Sorted by
Tagged with
56 views

### Can the likelihood ratio estimate multivariate confidence levels?

Wilks' theorem describes the log-ratio between the highest likelihood of a distribution $\mathcal{L}$ (aka the dominant mode, given at $\vec{x}_{m}$) and the likelihood of a distribution at a given ...
28 views

### Standard errors for Composite Marginal Likelihood

I am estimating a multivariate ordered probit model using a composite marginal likelihood (CML) approach. In other words, I replace the full likelihood function by a surrogate likelihood constructed ...
82 views

### Square root of an almost diagonal matrix

Is there an efficient way to compute square root of an almost diagonal symmetric Hessian matrix, which is diagonal with the exception of the last two columns and last two rows? Could the efficient ...
48 views

### AutoGrad Hessian in custom loss function in XGboost takes very long time

I am using the autograd package and generating the hessian of my loss function using that package as part of my custom loss function XGB model. However, it takes an extremely long time to iterate ...
89 views

### Calculate the uncertainty of a MLE

I have minimized the negative LL of a Poisson distribution to get an MLE of three parameters using scipy.minimize w/ Nelder-Mead. I want to calculate the uncertainty of the MLE. From reading, I ...
48 views

### Neural networks: why don't we use a multi-dimensional learning rate

I've searched a bit on the internet a have found the answer nowhere so I decided to post here. When confronted to an optimization problem, we know that the sanity of the problem can be characterized ...
96 views

### Saddle-free Newton method for SGD - while Newton attracts saddles, is it worth to actively replel them?

While 2nd order methods have many advantages, e.g. natural gradient (e.g. in L-BFGS) attracts to close zero gradient point, which is usually saddle. Other try to pretend that our very non-convex ...
202 views

### Hessian of logistic loss - when $y \in \{-1, 1\}$

Logistic Regression has two possible formulations depending on how we select the target variable: $y \in \{0,1\}$ or $y \in \{-1,1\}$. This question discusses the derivation of Hessian of the loss ...
35 views

96 views

### Should Bayesian estimated error smaller than MLE?

I am dealing with a fitting problem. Specifically, I am fitting a Lorentzian profile to the power spectrum of an solar-like oscillating star. Three parameters in the Lorentzian profile characterize ...
379 views

### Multiclass: I want to develop a customized objective function with weights given by both label and prediction, for Xgboost

I want to develop a customized objective function with weights given by both label and prediction, for Xgboost. Example, let's say you have 2 classes I want to assign a penalties according to this ...
168 views

### Cramér–Rao bound to multiple parameters

I was reading Cramér–Rao bound to multiple parameters from Wikipedia page, but I could not follow this line in the article: Let $\displaystyle {\boldsymbol {T}}(X)$ be an estimator of any ...
2k views

### Why not use the third derivative for numerical optimization?

If Hessians are so good for optimization (see e.g. Newton's method), why stop there? Let's use the third, fourth, fifth, and sixth derivatives? Why not?
14k views

### Explanation of min_child_weight in xgboost algorithm

The definition of the min_child_weight parameter in xgboost is given as the: minimum sum of instance weight (hessian) needed in a child. If the tree partition step results in a leaf node with the ...
1k views

### Gradient and hessian of the MAPE

I want to use MAPE(Mean Absolute Percentage Error) as my loss function. ...
2k views

### How does the second derivative inform an update step in Gradient Descent?

I was reading the deep learning book by Begnio, Goodfellow and Courville and there was one section where they explain the second derivative that I don't understand (section 4.31): The second ...
42 views

### Non-linear Likelihood function, large estimated standard errors

I have a highly non-linear (lots of jumps) likelihood function with K parameters (For example, a marked Hawkes Process used in seismology study). I implemented the L-BFGS-B optimization routine and it ...
78 views

### Pearlmutter's method for Hessian multiplication

I am trying to understand the abstract below from Pearlmutter's paper. Can someone clarify to me why $R_{\bf{v}}\{\bf{w}\}=\bf{v}$? Thanks a lot!
90 views

### Fast multiplication by the Hessian in Neural networks

I have question about the $R\{.\}$ function in Bishop's book on page 254 (see snippet below). My questions are as follows: I assume $R\{\bf w\}$ in (5.97) is the premultiplication of $\bf{v}^{T}$ ...