Confusion about parameter covariance using least squares method I am using the method of least squares to estimate parameter values for a nonlinear model with three parameters: $a$, $b$, and $c$. Call the sum of the squares of the residuals $\chi^2$. I plot $\chi^2$ as a function of $b$ and $c$ at the minimizing value of $a=\hat{a}$. For a particular data set, this looks as follows:

Naively, I would say that $b$ and $c$ are negatively correlated; e.g., it looks like if one is increased, the other must decrease to "keep $\chi^2$ small". However, a numerical estimate of the covariance matrix says that $\mathrm{cov}(b, c) > 0$. (I have calculated this a number of ways and the sign is always positive.) I am confused by this.
I think my misunderstanding may have something to do with the law of total covariance. That is:
$$\mathrm{cov}(b, c) = \mathrm{E}[\mathrm{cov}(b, c | a)] + \mathrm{cov}[\mathrm{E}(b|a), \mathrm{E}(c|a)]$$
Perhaps the plot I have made is only showing me $\mathrm{E}[\mathrm{cov}(b, c | a)]$ and the other term on the right-hand-side is positive and larger in magnitude, resulting in $\mathrm{cov}(b, c) > 0$.
Can someone verify if this is (or isn't) the case, and perhaps shed a little light on why one cannot (or can) look at the above plot the way I am trying to?
 A: I think I have found the answer: no, one may not look at the above plot and try to "read off" the sign of the covariance. It is more complicated than that. If the data are Gaussian, then $\chi^2$ is connected to the log-likelihood function as
$$\chi^2(\theta) = -2 \, \mathrm{ln} L(\theta) + \mathrm{constant}$$
In the limit of infinite data, the likelihood function takes on the form of a normal distribution in the fit parameters $\theta$. In the special case of three fit parameters, this is the trivariate normal distribution. The argument of the exponential can be viewed as describing various ellipses, and one can work out the angles their major axes make to the axis of interest. For the special case of unit variance and zero mean, the ellipse in the $\theta_1 - \theta_2$ plane has angle
$$ \phi = \tan^{-1} \left[ \frac{(\rho_{23}^2-\rho_{13}^2) - \sqrt{(\rho_{23}-\rho_{13})^2 + 4(\rho_{13}\rho_{23}-\rho_{12})^2} }{2(\rho_{13}\rho_{23}-\rho_{12})}\right]$$
where the $\rho_{ij}$ are the correlations. (Identical to the covariance in this case.) For plots of $\phi$ in the $\rho_{13} - \rho_{23}$ plane at various values of $\rho_{12}$, one can see that the angle of the ellipse changes sign depending on the values of $\rho_{13}$ and $\rho_{23}$.
