3
$\begingroup$

Suppose one has data that is suspected to obey a normal distribution. One computes a histogram of the data, and performs Pearson's Chi-Squared Test. To perform this test, one must compare the observed counts from each bin in the histogram against the expected counts in the corresponding bins by the formula below:

Pearson's Chi-Squared Formula

To compute the expected counts, one integrates the probability density function of the normal distribution from the left bin-edge to the right bin-edge for all bins using fixed parameters mu and sigma. The smallest Chi-Squared statistic (assuming the associated pvalue is not too high or too low) should correspond to the fixed parameters that best fit the data.

(If anything is incorrect, please correct me).

I have read that a better optimization technique is to use the cdf instead of the pdf. If one still wanted to use the cdf (instead of the pdf) in conjunction with Pearson's Chi-Squared Test to optimize the distribution fit, how would one go about this?

My initial thought is that the histogram of observations should be converted into a cumulative histogram, such that the expected counts can be obtained from the integration of the cdf instead of the pdf. Is this wrong?

(I am asking because I have a lognormal dataset; the Chi-Square fit of looks great but the pvalue is small enough to be approximated as zero. Any advice or corrections regarding misunderstood information would be appreciated).

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Chi-square testing goes back to 1900 and wasn't the method of choice even then for assessing fit to a normal distribution, as normal quantile plots (modern terminology) were already in use. Either way, binning discards information -- and in an arbitrary way. $\endgroup$
    – Nick Cox
    Commented Jan 9, 2022 at 18:25
  • $\begingroup$ It's unclear what is being "optimized." Moreover, since there's no difference whatsoever between a chi-squared test of a lognormal distribution and a chi-squared test of a normal distribution of the exponentials of the data, it looks like the underlying issue concerns understanding the p-values of distribution tests. $\endgroup$
    – whuber ♦
    Commented May 22, 2023 at 19:36

1 Answer 1

Reset to default
-1
$\begingroup$

CDF is the integral of PDF from negative infinity. The CDF values don't tell you the expected counts in each bin because as mentioned, it starts the integration from negative infinity.

You will need to apply integration for each bin endpoint.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ The integration starts at negative infinity, but I think the CDF will be zero from -inf until the first bin. Are you saying that to apply this Chi-Squared Test with the CDF, I would integrate the first bin from -inf to the first right-side bin-edge, would integrate the second bin from -inf to the second right-side bin-edge, etc. ? $\endgroup$
    – user146123
    Commented Oct 5, 2018 at 5:44
  • $\begingroup$ The CDF is specifically constructed to give probabilities, and therefore expected counts, within any interval whatsoever! $\endgroup$
    – whuber ♦
    Commented May 22, 2023 at 19:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.