I have a (pretty noisy) 1D array of data in Python that I fit a linear function to -- see plot below. The x-axis is a wavelength scale. enter image description here

I would like to measure the "goodness of fit" of this new line to my actual data at multiple different intervals in my data, because I'm trying to see if it's a worse fit in a particular region over another.

I started off by just computing the chi squared value with SciPy in different regions:

chisq = scipy.stats.chisquare(data, fit_line) 

But I got negative values, which doesn't make sense in terms of a chi squared value... however this arises because my data (and hence best fit line) is all negative. I then came across the answer here regarding the R^2 approach, but I do not know how to interpret this.

Also, the different regions I want to calculate this for have different length arrays, so I'm wondering if my results will be skewed because of this. Can I just calculate (data-fit_line)**2 for each of my data points in the region, then maybe divide by the number of points in that region to normalize it? I do not understand why the formal chisquare function by scipy is different. What would the best measurement be for my case?

  • $\begingroup$ Why not just do a linear regression? The Statsmodels library comes with a whole bunch of statistics about the fit. $\endgroup$ – Demetri Pananos May 9 '19 at 20:15

So first off, that isn't how the the chi-square function should be used.

If you want to know the "goodness of fit", use the R squared stat. R squared tells you how much of the observed variance in the outcome is explained by the input. Here is an example in python.

import numpy as np
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt
from sklearn.metrics import r2_score
#Your data
N = 1000
x = np.random.normal(size = N)
y = 2*x+1 + np.random.normal(size = N)

#Your model

#Your statsitic
r2_score(y, model.predict(x.reshape(-1,1)))

This returns 0.801, so 80.1% percent of the variance in y seems to be explained by x. The higher this number the better.

As for assessing fit over different regions, I think you can look at the residuals of your model and see if there are patterns or heterogeneity of variance.

| cite | improve this answer | |
  • 2
    $\begingroup$ I don't think $R^2$ will answer this question, for two reasons: first, it doesn't measure goodness of fit in the sense described by the question; and second, it won't provide the local information that is requested. $\endgroup$ – whuber May 9 '19 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.