# Covariances and correlations in curve fitting

I have a set of data that I am trying to curve fit and I'm ultimately interested in the errors on my fit coefficients. I take my errors on each fit coefficient as the on-diagonal elements of the covariance matrix and scale by the residual variance. The covariances between fit parameters are still large, and intuitively I feel as if this should somehow affect the error in my fitted model. The question I am hoping to get answered is "How should I inflate my error bars to take into account non-negligible covariances in my curve fit?"

One thought that I had (and let me know if this is incorrect) was that perhaps I could get around this by diagonalizing my covariance matrix. In my new, rotated basis, my new coefficients are now admixtures of the old ones. i.e. if my covariance matrix is

$$cov = \left(\begin{array}{ccc} 2 & 0.5 & 0.1 \\ 0.5 & 4 & 0.2 \\ 0.1 & 0.2 & 3.5 \end{array}\right)$$ , my coefficients $$a_1, a_2, a_3$$ (with variances $$cov_{11},cov_{22},cov_{33}$$ respectively) will now turn into new coefficients $$\tilde{a_i}=c_1a_1+c_2a_2+c_3a_3$$, where coefficients $$c_i$$ are determined by the eigenvectors of $$cov$$. Covariances between the fit parameters are now removed, but I have a more complicated form of my actual fit parameters. If I want to use my fitted model and know the associated value and error for a given value of my independent variable, I can get my errors from the diagonalized $$cov$$, but I have to use parameters $$\tilde{a_1},\tilde{a_2},\tilde{a_3}$$.

Not only does this seem unwieldy, I don't even see how this could physically make sense if my parameters have different units (e.g. $$a_1, a_2, a_3$$ are the coefficients for different order polynomial terms - they each will have units in different powers of my independent variable). In principle I could bootstrap my data, fit each dataset, and then get parameter uncertainties by considering the distribution of fit parameters. The issue with this is that my dataset is small, I only have 5 points and I'm not sure how effective a bootstrap would be here.