3
$\begingroup$

Hi I am fit a maxwell distribution and attempt to find the chi squared value in two cases:

  1. When the data is normalized.
  2. When the data is un-normalized.

My problem is that the two different cases give completely different values of chi-squared! So my question is: is it mathematically correct to use the chi squared test on normalized data?

Notes on my data:

Number of counts: 41,000

Expected fit: Maxwell distribution.

Method to calculate Chi-squared: Find the best maxwell fit to data, then use scipy.stats.chisquare function in python programming language to calculate the chi-squared value using the experimental and expected counts.

Chi-squared documentation For reference: http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.chisquare.html#scipy.stats.chisquare

I have attached the two graphs here for you to see the difference in chi squared values:

  1. The un-normalized graph: please ignore the y-axis which says "Normalized" Please ignore the "Normalized" on the y-axis here

  2. The un-normalized graph: enter image description here

$\endgroup$
  • 1
    $\begingroup$ Please explain how you are conducting a chi-squared test. Ordinarily (in this goodness-of-fit context) it is conducted on counts, but since your "un-normalized graph" evidently contains no counts you must be doing something different. Your description of that process will help people provide good answers to this question. $\endgroup$ – whuber Sep 9 '14 at 17:36
  • $\begingroup$ Please note that the top plot is the un-normalized plot with counts upto almost 10,000 in the most populated bin. The bottom plot is the normalized plot. I hope that was not what confused you. To answer your question, I am simply counting from a list of 41,000 numbers and binning hem into a histogram. I fit a maxwell distribution (I expect that distribution) on the plot and calculate the chi-squared from that. Does that make it clear? $\endgroup$ – Abhinav Kumar Sep 9 '14 at 17:39
  • $\begingroup$ I doubt what you are doing is the best way to go. Although written in the context of testing if a distribution is normal (instead of Maxwell) you should read the following thread: Is normality testing 'essentially useless'? $\endgroup$ – gung Sep 9 '14 at 17:43
  • $\begingroup$ I'm afraid it's still unclear. I don't think I'm confused (yet): I'm just uninformed because you haven't provided the information needed to understand what calculations you are performing with the second plot. $\endgroup$ – whuber Sep 9 '14 at 17:53
  • $\begingroup$ @whuber : In the second plot, I have have normalized the histogram meaning that the area under the curve is 1. Or in other words, the area in the histogram is 1. I apply the same chi-squared test on the second plot as I did on the first plot (please read my new edit on the code about how I do that). The test return a p-value which I write on top of the plot. Please disregard popt mentioned on the plot. Please tell me what other information I can give you. Thanks! $\endgroup$ – Abhinav Kumar Sep 9 '14 at 18:14
5
$\begingroup$

What the $\chi^2$ goodness-of-fit test is, how to determine whether it is applicable, and how to apply it are explained in great detail elsewhere on this site, so let's cut to the chase: the statistic is given by tabulating

$$\chi^2 = \frac{(O_i-E_i)^2}{E_i}$$

where $(O_i, E_i)$ are the observed and expected counts, respectively, in the histogram bins indexed by $i$.

What happens when you rescale the counts, such as when normalizing a histogram? They all get changed by a constant $\lambda$ and so, therefore, do the expectations. The new value of the statistic is

$$\frac{(\lambda O_i-\lambda E_i)^2}{\lambda E_i} = \lambda \frac{(O_i-E_i)^2}{E_i} = \lambda \chi^2.$$

The value of the statistic changes. Thus you have two options:

  1. Don't do this.

  2. Divide the new statistic by $\lambda$ to recover the correct $\chi^2$ statistic.


We can reverse-engineer the problem from the graphics and information helpfully posted in the question. Because there are $41000$ data and a bin width of $45$ km/s, the second graphic must have been obtained by multiplying the counts in the first by $\lambda \approx 1/(41000\times 45) = 5.42\times 10^{-6}$. As a check, that should have changed the highest bar in the first histogram from around $9500$ to about $0.0051$ in the lower--and that looks exactly right. Thus, the original $\chi^2$ statistic of $85.06$ (reported at the top of the first graphic) was changed (incorrectly) to $85.06\times 5.5\times 10^{-6}=0.0000461$, exactly as reported at the top of the second graphic.

Although no information is supplied about the degrees of freedom (the number of bins used), we can ascertain that it is well below $85$ (for otherwise the original $\chi^2$ statistic would have a p-value measurably greater than the reported value of $0.000$). Moreover, the df must be at least $1$. It's likely a little less than the number of bins spanning the histograms, of which there are between $10$ and $20$. For any $\chi^2$ distribution with df in that range, a value of $0.0000461$ or greater is almost certain, resulting in the reported p-value of $1.000$ in the lower graphic.

Figure

Normalizing the data changed the $\chi^2$ statistic from $85$ to $4.6\times 10^{-5}$. The p-value is the area under a curve like this one to the right of the statistic. That area shifted from essentially zero to essentially one, completely changing the answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.