# Parameter uncertainity in least squares optimization: rescaling Hessian

Given a least squares optimization problem of the form:

$$C(\lambda) = \sum_i ||y_i - f(x_i, \lambda)||^2$$

I have found in multiple questions/answers (e.g. here) that an estimate for the covariance of the parameters can be computed from the inverse rescaled Hessian at the minimum point:

$$\mathrm{cov}(\hat\lambda) = \hat H^{-1} \hat\sigma_r^2 = \hat H^{-1} \frac{\sum_i ||y_i - f(x_i, \hat\lambda)||^2}{N_{DOF}}$$

While I understand why the covariance is related to the inverse Hessian (Fisher information), I haven't found anywhere a demonstration or explanation for the $$\hat\sigma_r^2$$ term, although it appears reasonable to me on intuitive grounds.

Could anybody explain the need for the rescaling by the residual variance and/or provide a reference?

In principle it's the stopped series expansion of the term that is minimized. So, if one stops at the second term, the expression is in the simple 1D case around the (minimum) value of a: $$f(x) = f(a) + f^\prime(a)(x-a) + \frac{1}{2}f^{\prime\prime}(a)(x-a)^2$$ If you change to the vectorized version, the derivatives are replaced by their respective vectorized parts.