0
$\begingroup$

I have a data set of $N$ points to which I have fit an equation of $n$ parameters $\theta_{1..n}$ such that $y_i \sim f(x_i; \theta_{1..n})$. These data $(x_{1..N},y_{1..N})$ have been provided with standard deviations $\sigma_{1..N}$ after having been acquired several times. My nonlinear regression took this into account by minimizing the weighted RSS, where the reciprocal variances ${\sigma_{1..N}}^{-2}$ were used as weights. I now want to estimate the confidence intervals and covariance for parameters $\theta_{1..n}$.

I would have liked to resort to bootstrapping, but the process of resampling is too computationally expensive for the function $f$ being used. Instead, I suppose I would like to fall back on estimating the (co)variance matrix using the Jacobian. In the case of an unweighted covariance matrix, I know this can be done (assuming $J^\mathsf{T}J$ is invertible) as $$ V(\theta_{1..n}) = \sigma^2 \left(J(\theta_{1..n})^\mathsf{T} J(\theta_{1..n})\right)^{-1} $$ for $n\times n$ covariance matrix $V$, variance $\sigma^2=\frac{1}{N-n} \sum_{i=1}^N \left(y_i - f(x_i; \theta_{1..n})\right)^2$, $N\times n$ Jacobian matrix $J$ (entrywise $J_{ij}=\frac{\partial}{\partial\theta_j}f(x_i;\theta_{1..n})$), and $\theta_{1..n}$ the optimized parameters. Here, I can estimate the Jacobian with numerical differentiation, allowing evaluation of $f$ only a couple of times per parameter $\theta$ instead of hundreds/thousands of time as in bootstrapping. How can I adapt this formula to estimate a weighted covariance matrix? Is it as simple as using weighted $\sigma^2$ and $J$? If so, how does one calculate a ``weighted'' Jacobian matrix?

Alternatively, is there a more appropriate way to approach such an issue for an expensive function $f$? I'd like to make use of the error bars on the data points to their maximum extent to inform the error bars on the fitted parameters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.