3
$\begingroup$

I have been doing some fitting using the fairly standard expression, $$\chi^2 = \sum_i \frac{(y_{i} - F(x_{i}, \theta))^2}{\sigma_{i}^{2}}$$ where y is my measured data, $\sigma$ is the experimental uncertainty and $\theta$ is the parameter I'm estimating.

I know that in reality my data are not independent and that I can have correlated errors. Is there a standard way of developing a more detailed model taking into account errors and correlated errors?

I have found some hints from high energy physics such as http://www.desy.de/~blobel/banff.pdf .

$\endgroup$
3
$\begingroup$

Yes, there are many methods for dealing with correlated data errors.

This is usually done by modelling the likelihood function, $p(D|\vec{\theta})$, i.e., modelling how the data was sampled as a function of the parameters $\vec{\theta}$ which include not only the parameters of your model ($F(x_i,\theta)$), but the parameters that generate the correlations. For example, a model for the likelihood could be the multivariate gaussian distribution, i.e., \begin{equation*} p(D|\vec{\theta})=\frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}}\text{exp}\left(-\frac{1}{2}[\vec{y}-\vec{F}(\vec{x},\theta)]^T\Sigma^{-1}[\vec{y}-\vec{F}(\vec{x},\theta)]\right)\text{,} \end{equation*} which can be thought of as the most conservative distribution to use (in a maximum entropy sense) where $\theta\in\vec{\theta}$ are the parameters of your deterministic model $F$, $\Sigma\in\vec{\theta}$ is the covariance matrix (which will model the correlation between your data points), $\vec{y}$ is the vector containing the observations (i.e. the $i$th element is $y_i$) and $\vec{F}(\vec{x},\theta)$ the corresponding values of the parametric model you are trying to fit (i.e. the $i$th value is $F(x_i,\theta)$). The special case you mention (for uncorrelated errors) can be derived by turning $\Sigma$ into a diagonal matrix (with elements $\Sigma_{i,j}=\delta_{i,j}\sigma_i^2$) and maximizing $p(D|\vec{\theta})$ (which in turn will minimize the equation that you cite).

How to model the covariance matrix $\Sigma$ in this case is the real problem, and will vary between data sets or applications. For example, in time series analysis one can model correlation between residuals as order $p$ autoregressive procceses (which are, in a sense, "maximum entropy" procceses), which for order $p=1$ model the elements of the covariance matrix with only two parameters $\alpha$ and $\sigma$, \begin{equation*} \Sigma_{i,j}=\frac{\sigma^2}{1-\alpha^2}\alpha^{|j-i|}\text{.} \end{equation*} There are other models (both for the likelihood function and for the modelling of the covariance matrix), however, and you'll have to search for the one to fit your needs. In general, an analysis to the autocorrelation function of your residuals with the model you cited will give you advice on what kind of model you need. If you need help in those subjects, please consult any time series analysis textbook such as the one by Brockwell & Davis. Another good reference when modelling flicker noise (which arises frequently in physics) is the wavelet method of Carter & Winn (2009).

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.