# What goodness-of-fit tests avoids the ambiguity in chi-square statistic?

The chi-square statistic is a fairly standard goodness-of-fit test defined as:

$$\chi^2 = \sum_k^N \left ( \frac{d_k-m_k}{\Delta d_k}\right )^2$$

where $$\mathbf{d}$$ is a vector of data with error $$\Delta \mathbf{d}$$, and $$\mathbf{m}$$ is some modelled data. All vectors have length $$N$$.

This definition ensures that if, on average, the modelled data are within the error bars, then $$\chi^2 < N$$ and if the modelled data are outside the error bars then $$\chi^2 > N$$.

The problem with this test is that there is an inherent ambiguity to the test. See this example:

In the above figure, the data are shown as black dots with errors bars and there are two different models shown as red and blue dots. As you can see, both of these models will return $$\chi^2 = N$$.

What is another type of goodness-of-fit test which will show that both of these models fit the data equally well while simultaneously showing that the models themselves are different?

Any help or ideas are appreciated.

• You would need a statistic that returned more than one value at once. COnsider two further models one which alternated being above and below and the other which alternated the other way. There are may ways to both differ from the data and from each other. Commented Nov 23, 2018 at 4:02

You need to decide what exactly are you measuring. $$\chi^2$$ test measures the squared errors. Obviously, after squaring it loses the information about the direction of an error. If the direction was important then you shouldn't be squaring and using this particular test. For instance, you could simply measure the mean error, and in both cases it would be different than zero. For one model it would negative, and positive for another. You could meausue whether the biases are significant, then make a conclusion about it.