When doing parameter fits with mathematics frameworks as e.g. numpy, often a covariance matrix is returned. I wonder how to interpret these and if the following is right:

The entries of the covariance matrix after an optimization show:

  • on the diagonal: the variance of the found parameters:
    • $Cov_{1,1}$ is large -> changing parameter 1 has no big impact on the quality of the fit, "the fitter is not really sure about this parameter"
  • on the other elements:

    • if $Cov_{1,2}=Cov_{2,1}$ are large, parameter 1 or parameter 2 could be removed from the model function as it is expressible by the other one.
  • often a strong covariance of two parameters comes with a strong variance too as many different parameter pairs can be found

  • $\begingroup$ Could you explain your criteria for determining when the coefficients of the covariance matrix are "large"? $\endgroup$ – whuber Dec 12 '18 at 18:38
  • $\begingroup$ @whuber: with "large" I mean values that are large compared to other covariance matrix entries or for the diagonal elements one can do the square root to have a measure of standard deviation that is in the same unit as the parameter and thus can be compared to it. $\endgroup$ – user2722085 Dec 13 '18 at 9:03
  • $\begingroup$ That sense of "large" has little meaning because it makes no comparison to the estimated values of the parameters. For instance, suppose a parameter whose units are in meters has a variance of 100 (i.e., a standard deviation of 10 meters). Compared to an estimated distance of 10,000 kilometers that variance is truly small, but compared to an estimated distance of 1 meter it is huge. $\endgroup$ – whuber Dec 13 '18 at 15:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.