3
$\begingroup$

I have a data set of 903 continuous observations, that I graphically visualize with a histogram. The bin and width values could be optimized, but it is logical from the distribution that I have a Gaussian function.

When I do the fitting, I use the frequency values of the data as the Y-values. For example, if the observations are ${(2,3,3,3,4,4,5)}$, and the user-defined bin width is 1.0, then the corresponding y-values would be $(1,3,3,3,2,2,1)$, assuming that the first bin limit will be assigned as the minimum value.

I am not obtaining statistical significance with a Gaussian fitting, in terms of goodness of fit ($Q$) with the Chi-squared test ($\chi^2$). In other words, my null hypothesis is rejected making that a Gaussian model does not represent the experimental data.

Now, I am doing the same test but using an averaged shifted histogram version of my x-values and the frequency for those averaged bins as the y-values for each observation, I now obtain good results in terms of goodness of fit.

I need to clarify if it is valid to realize a Goodness of fit for an averaged shifted histogram, or if there is clear bias for data overfitting.

Here it is a q-q plot of the data:

QQ Plot

Improve this question
$\endgroup$
10
  • 1
    $\begingroup$ (1) What does a q-q plot show? (2) What is the reason for testing a Gaussian distribution, anyway? Very few (if any at all) statistical tests or models require data to be so close to Gaussian that they will pass a Goodness of Fit test using 700 iid values. $\endgroup$
    – whuber ♦
    Commented Mar 11, 2013 at 23:26
  • $\begingroup$ Upload your q-q plot anywhere and link to it from here. But that might not be relevant: what is your answer to the second question in my comment? $\endgroup$
    – whuber ♦
    Commented Mar 12, 2013 at 3:35
  • $\begingroup$ postimage.org/image/ct53wuqiv This is the q-q plot $\endgroup$
    – ilibarra
    Commented Mar 12, 2013 at 3:40
  • $\begingroup$ (2) The reason of Gaussians is that we would like to be based on simple mathematical functions as much as possible to describe our experimental data. We have gradient optimization algorithms that are slower, in terms of total calculation time, if we end up fitting the functions with splines or similars. $\endgroup$
    – ilibarra
    Commented Mar 12, 2013 at 3:43
  • $\begingroup$ Thanks--I have embedded your plot in the question. But it shows only about 120 values--what happened to the 700 values you describe? Your plot shows that about six to seven percent of all values are too high to be consistent with a Normal distribution: if any test fails to reject the hypothesis of normality, it's a really bad test! BTW, the Web link in your question doesn't go anywhere relevant: there seems something the matter with it. $\endgroup$
    – whuber ♦
    Commented Mar 12, 2013 at 3:45

1 Answer 1

Reset to default
3
$\begingroup$

The biggest problem is that an averaged shifted histogram has positive dependence in adjacent bins, so a test derived on an independence assumption (aside the negative dependence induced by the total count being conditioned on, which is adjusted for) won't have the right distribution for its test statistic.

It's possible to adapt a test for such dependence, but the vanilla version of the test will be wrong.

[If you want to test for normality, doing it from a histogram isn't a particularly good way to do it. A Shapiro-Wilk or Shapiro-Francia test, an Anderson-Darling test, or perhaps a Smooth test of the kind discussed in Rayner and Best's book Smooth Tests of Goodness of Fit would be better. The nice thing about a Shapiro-Francia test is it's just based on the correlation in a normal scores plot (Q-Q plot for normality), which gives a visual assessment of the non-normality]

--

Edit - looking at your QQ plot - the data are very far from normal. No reasonable test would fail to reject normality at that sample size. A Lilliefors test or an Anderson-Darling or a Shapiro-Wlik or a smooth test with a standard number of terms ($k=4$ or $k=6$) will all reject easily... you don't even need to test that.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for the reply. I needed to confirm about the ASH bias. The reason of using a Chi-squared test is because I currently have several distributions that are being fitted using the Levenberg-Marquardt algorithm, and not necessarily to a normal distributions. Sometimes I obtain polygaussians, with two or three peaks. I'll check the Shapiro-Wilk test for the possible mono Gaussian distributions. Regards. $\endgroup$
    – ilibarra
    Commented Mar 11, 2013 at 22:33
  • $\begingroup$ For bias it kind of depends what bias you mean - you get bias with any smoothing, and ASH is smoothing (for that matter, so are plain histograms). For more general distributions, the Anderson-Darling test is still going to generally be better than chi-square, though its null distribution needs to be adjusted for parameter estimation (on the other hand, if I recall correctly, even so, it tends to converge fairly quickly so in large samples it may not matter much) $\endgroup$
    – Glen_b
    Commented Mar 11, 2013 at 23:02
  • $\begingroup$ On the adjustment for parameters estimation in the Anderson Darling test, if you were inclined to pursue that further, the book by D'Agostino and Stephens, Goodness-of-Fit Techniques, discusses it in some detail. (On the other hand, usually formal testing of goodness of fit usually doesn't make much sense.) $\endgroup$
    – Glen_b
    Commented Mar 11, 2013 at 23:13
  • $\begingroup$ I would like to thank you for all your comments. I have found the book "Smooth Tests of goodness of Fit" and now I am looking for the second book. I will digest a bit all this information and I will reply again. $\endgroup$
    – ilibarra
    Commented Mar 12, 2013 at 2:24
  • $\begingroup$ I've made an edit to my answer $\endgroup$
    – Glen_b
    Commented Mar 12, 2013 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.