I have an astrophysical non-linear curve, specifically a power spectrum. I need to fit this curve with a model and obtain the goodness-of-fit (GOF). This gives me expected and observed values. The data also have observational errors related to instrumental uncertainties. Is there any way I can integrate observational errors with a chi-square test (or any other GOF test)?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ Welcome to our site! I have taken the liberty of making your question slightly more flexible--so it is not tied entirely to the chi-squared test--in case any respondents would care to suggest other tests that might better meet your needs. $\endgroup$– whuber ♦Apr 18, 2014 at 15:50
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Are you using some package to fit the data or is this a school problem where you are writing a non-linear least squares code? Just about every package allows you to input the errors on the data points and it will output the errors on the fitted parameters or full covariance matrix. In fact, you should be able to input the covariance matrix of errors (if you know it) but power spectra usually have fairly diagonal covariance matrices.
The basic idea is to look at weighted least squares. You don't want to just ignore the errors when doing the fit and then use them to calculate the error. The fitter needs the errors in order to optimize the fit.
If the errors are small enough, the distribution of errors may be reasonably well described as multivariate normal. The terms in the output covariance matrix actually comes from linearizing the model around the best fit solution.
If you really want to characterize the error distribution fully, you'll have to go to something like markov chain monte carlo.
-
$\begingroup$ I am using QDP to fit the data for my Msc. Thesis. Fitting is not the problem. After I obtain two fits, I have to derive a p-value to show that the second fit is better. F-test accepts as input chi sqr values in IDL package that I am using. However the problem is this: I could not find any reasonable way to add observation errors into calculation of chi sqr. $\endgroup$ Apr 21, 2014 at 10:17
-
$\begingroup$ A researcher at NASA advised us to modify chi sqr test as follows: instead of dividing by variance, change the summation by dividing (o-e)^2/i^2 where o are observations, e expected values and i observation errors. This doesn't seem to follow from either Pearson's test or dividing by variance. How must I calculate chi-square in this case? $\endgroup$ Apr 21, 2014 at 10:17