New answers tagged

0 votes

Is Hessian of neural nets with NLL loss positive semi-definite?

Hessian of the loss is defined over data distribution whereas Fisher information is defined over model's distribution: $$ \begin{aligned} H(\theta) &= E_{x \sim Data}[-\nabla_\theta^2\log p_\...
user avatar
2 votes
Accepted

Which form of Jeffrey's prior can be used for a three-parameter distribution?

The Jeffreys prior for a three parameter model is the square root of the determinant of the Fisher information matrix. If you take the product of individual Jeffreys priors, this is called the "...
user avatar
  • 36

Top 50 recent answers are included