5
$\begingroup$

I am using a model to calculate observed frequencies, which some times gives non-integer values. I can round these frequencies but that seems like artificially distorting the information I have. For Example:

Example Data
          Yes        No
Male       11        19
Female     16        17

Assume my model just divides everything by 3, so model data becomes:

           Yes        No
Male       3.67        6.33
Female     5.33        5.67

This data has to be used as "observed frequencies". Doing a chi-square test gives p value of 0.58. However, if I round this data to integers, chi-square test will give a p value of 0.8, which is very much different. My question is: is chi-square test theoretically valid on non-integer observed frequencies?

Edit: Please note that data and model specified in the question are not real, just to make you understand the problem I am facing. Real data is of this type.

            Male     Female
Source1     10.8      18.2
Source2     16        17

The real data is the prediction of males and females according to the job roles and City from Bureau of Labor Statistics.
I have no control over the data coming from source1, which (surprisingly) contains decimal point numbers. All I can do is round the data from source1.

$\endgroup$
  • 2
    $\begingroup$ You need to explain in more detail how the non-integer values arise. How do you come to be dividing by 3? Why do you do that? [If you do divide counts by some constant, the test no longer works the way it originally would, because the denominator in the sum is no longer adjusting correctly for the variance of the numerator] $\endgroup$ – Glen_b Oct 28 '16 at 23:23
  • $\begingroup$ @Glen_b, This is just an example to explain that my observed count have decimal points. Dividing by 3 is not the real operation. This is neither real data, nor real model. The real data is the prediction of males and females according to the job roles and City from Bureau of Labor Statistics which (surprisingly) give prediction in decimal points. I do not want to change that. I also have similar data from other source (Integers), and I want to compare if both the sources give similar distribution. So I want to use chi-square test, however I do not want to distort BLS predictions by rounding. $\endgroup$ – Gaurav Singhal Oct 29 '16 at 10:31
  • $\begingroup$ I should have specified that in question itself probably. Also, apologies, if the above comment seems rude, I respect you a lot (for the great posts you have put on the site) and just wanted to save space as I was going out of the character limits. $\endgroup$ – Gaurav Singhal Oct 29 '16 at 10:34
  • 1
    $\begingroup$ Predictions don't have the same properties (including the same uncertainty) as count data. Your test won't work - you can't just pretend predictions are observed counts. $\endgroup$ – Glen_b Oct 29 '16 at 10:37
  • $\begingroup$ OK, I believe then I need to ask a separate question on how to solve my problem. However may I request you to have a look on the edited question and respond if possible. $\endgroup$ – Gaurav Singhal Oct 29 '16 at 10:42
3
$\begingroup$

observed count have decimal points.

If you have fractions, you don't have observed counts but something else. Counts actually count things, 0, 1, 2...

. The real data is the prediction of males and females according to the job roles and City from Bureau of Labor Statistics

Predictions don't have the same properties (including the same uncertainty) as count data.

The chi-squared test relies on the data being actual observed counts, not predictions of counts or any other manipulation of counts. This is needed to obtain the correct scaling of $O_i-E_i$ (the denominator is based on particular assumptions that in general won't hold for things that are not counts).

As a result, your test won't work - you can't just treat predictions as observed counts. It's irrelevant whether they were rounded integers or not (the only difference of any consequence is that the non-integer values made it obvious you didn't have actual observed counts; if the predictions had been rounded you might never have known there was a problem).

$\endgroup$
1
$\begingroup$

Even before rounding, parts of your question describes things which seem to me problematic. IMHO, it pays to consider them, as they relate to the cause of rounding.


Scaling

Say my model just divides everything by 3

The rationale behind this test involves the multinomial distribution, and contains combinatorical terms of the form

$${n \choose n_1 \cdot n_k}$$

These terms are not invariant to scaling. I.e., you cannot replace this with

$${\alpha n \choose \alpha^k n_1 \cdot n_k} = {\alpha n \choose \alpha n_1 \cdot \alpha n_k}$$

and expect to get the same results.


Test Assumptions

It's possible your division of 3 is caused by this being an average of three observations. In this case, though, there's a problem with assuming that the test is relevant here:

A common rule is 5 or more in all cells of a 2-by-2 table

Post the division by 3, this does not hold, and the numbers are not in the range where it can be assumed that this test is applicable.

$\endgroup$
  • $\begingroup$ Thanks for the answer @Ami Tavory however, may I request you to read my comment on the Glen's comment. $\endgroup$ – Gaurav Singhal Oct 29 '16 at 10:35
  • 1
    $\begingroup$ @GauravSinghal If it's predictions, then the distribution is entirely different than that which is behind Chi square, and it doesn't seem like there's a justification for using it. If you'd like to test that predictions differ significantly from something else, I'd suggest you use a non-parametric test (e.g., Mann-Whitney). $\endgroup$ – Ami Tavory Oct 29 '16 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.