2
$\begingroup$

In probabilist statistics, the properties of a discrete Pearson's chi-squared test hold that:

\begin{aligned} \chi^2 = \sum_{i=1}^{r} \sum_{j=1}^{c} {(O_{i,j} - E_{i,j})^2 \over E_{i,j}} \end{aligned}

Where $\chi^2$ is our (Pearson's) cumulative test statistic, $O_{i,j}$ is an observed frequency in a given contingency table, $E_{i,j}$ is the an expected (theoretical) frequency for the same, and $r$ and $c$ are the number of rows and columns in the table, respectively.

Assuming this is true, a Pearson's chi-squared test for independence states:

\begin{aligned} E_{i,j}=\frac{\sum_{k=1}^c O_{i,k}\ \sum_{k=1}^r O_{k,j}}{N} \end{aligned}

... which is a mathematically terse way of saying that, for all observations in $O_{i,j}$, we expect a frequency derived from the row and column values.

I'm currently looking at generalizing this test for continuous variables estimable by PDF. Phrased differently: given the joint probability distribution of an intersecting function, I would like to be able to perform a chi-squared test for independence.

My intuition states that, for the continuous case, the first formula can be generalized as:

\begin{aligned} \chi^2 = \int_I \int_J {(\hat{O_{i,j}} - \hat{E_{i,j}})^2 \over \hat{E_{i,j}}}\ di\ dj \end{aligned}

Where $\chi^2$ remains our cumulative test statistic, $\hat{O_{i,j}}$ is an observed frequency off the joint probability distribution, given $i \in I$ and $j \in J$, $\hat{E_{i,j}}$ remains the expected frequency, and $r$ and $c$ briefly disappear. We then define $\hat{E_{i,j}}$ to a comparable function that looks something like:

\begin{aligned} \hat{E_{i,j}}=\frac{\int_c \hat{O_{i,c}}\ dc\ \int_r \hat{O_{r,j}}\ dr}{2} \end{aligned}

... where $r$ and $c$ are the partitions selected during integration by, for example, an adaptive quadrature. Note further that this relationship breaks down for infinite bounds, as $r$ or $c$ approach infinity (or conversely, as the value of any evaluation of $\hat{E_{i,j}}$ approaches zero).

This is intuitive only in the sense that these calculations are made practical by tools like SciPy, and I'm leaving out a lot of detail so as not to bore or confuse the reader.

But, I am curious: is method, as tersely presented, correct? I can find no guidance contradicting it. The goal of such a thing is to make terms expressible as continuous PDFs smoothed by a Gaussian kernel possible, for unorganized ordinal and nominal data.

Thank you in advance.

$\endgroup$
  • $\begingroup$ The integral itself is always $1$ for a continuous distribution, because the numerator equals $E_{i,j}$ almost everywhere (and presumably $E_{i,j}$ is the pdf of the joint distribution). Or perhaps it is intended in some special sense? $\endgroup$ – whuber Dec 15 '11 at 2:31
  • $\begingroup$ @whuber Did you means 'the numerator equals $E_{i,j}^2$ ...'? $\endgroup$ – onestop Dec 15 '11 at 12:26
  • $\begingroup$ @Onestop Sorry; I meant the integrand equals $E_{i,j}$. Thanks for catching that. $\endgroup$ – whuber Dec 15 '11 at 16:05
  • $\begingroup$ @whuber I updated the question to be more specific and to define certain qualities that need to hold true. Among them, the evaluation of $\int_I$ and $\int_J$ must not have infinite bounds, because as $E_{i,j}$ approaches 0, each infinitesimal evaluation approaches a singularity. It's a bit weird to think of integrals in this way, but then, this is how packages like QUADPACK evaluate them. -- As a bonus, I believe the properties of the gamma function bounding the error of the chi-squared test also hold true: mathworld.wolfram.com/Chi-SquaredTest.html $\endgroup$ – MrGomez Dec 15 '11 at 22:53
  • 1
    $\begingroup$ Thank you. I think you'll get a glimpse of what goes wrong if you consider a case where the kernel halfwidth is small. Your expression appears to reduce to a chi-square-like formula where every observation has been placed into its own bin. That will not have a chi-square distribution. It raises the additional question of what half-width to use for the kernel estimates, because your statistic (and its sampling distribution) will depend heavily on the choice of half-width. $\endgroup$ – whuber Dec 16 '11 at 14:45
2
$\begingroup$

In general, because continuous data usually has a dispersion attribute, the Pearson-Chi Square test doesn't make any sense. This is because the expected value in the denominator is attributed to the mean-variance relationship of categorical data. If you're looking for a generalized version of the Pearson chi-squared test statistic, you must specify what attribute of the chi-square statistic is interesting to you. Because it is the score test for a logistic regression model, you could consider using a score test for any other regression model, like linear regression, or use a different test statistic like those from the Wald or Likelihood ratio tests. This addresses the inferential aspect of the test, whether the row and column variables are independent.

Alternately, you may be interested in this test statistic as a measure of goodness of fit or calibration for a predictive model. If you can sensibly bin the data into distinct groups, you can still use the chi-square test statistic, or a kappa statistic measuring agreement between observed and predicted data, especially useful with more than 2 distinct groups in the outcome. I would prefer using the continuous scale and using the MSE as a measure of predictive accuracy, more preferrably the cross-validated MSE of your predictive model. This can also be standardized into a measure like the R^2 value which has associated significance tests as well.

$\endgroup$
  • $\begingroup$ The trouble with metrics and discretization is I'm looking to simultaneously compute the significance of a relationship and infer the shared properties of data of unknown distribution, noisiness, or disposition. Such is the trouble with what we're working with: a heap of unverified, non-normalized biological data of dubious quality. Ultimately, any observed relationship filtered to a particular significance will only tell part of the story. -- Your points are otherwise well taken. $\endgroup$ – MrGomez Dec 15 '11 at 0:52
  • $\begingroup$ It sounds like you may very well be interested in prediction then. Prediction can be very useful as part of a hierarchical modeling approach. Say you're running a clinical trial with several different sites, and each site is responsible for measuring ketones, bilirubin, eGFR, etc. weekly on individuals. Suppose further that some sites give very precise data, whereas others are shoddy and often missing. You can create a prediction to pool data across groups and improve the understanding of the lab parameters. This calls for standard reliability testing for continuous outcomes as I mentioned. $\endgroup$ – AdamO Dec 15 '11 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.