# Using Chi Square Test to check the effect of a variable

So I am a quite beginner in statistics and I am trying to see whether I am using a statistical method in a correct way or not.

I am trying to see the effect of income level towards fraud behavior. So in the so-called population, I have counts for each of the bad/good/new users:

BAD: 260
GOOD: 480
NEW:50


I also have a group of high-income users with these counts:

BAD: 50
GOOD: 95
NEW: 8


I am trying to see whether high income affects user fraudulent behavior. Can I use Chi-Square Goodness of Fit test in this problem? By using Chi-Square Goodness of Fit test, I am trying to see if the distribution of bad/good/new user with high income level differs significantly from the expected.

When I do this, my chi-square-stat does not exceed critical value and have high p-value (0.88) and thus I don't reject the null hypothesis (ie. there is not effect of income level towards user fraudulent behavior)

I would like to see if this is the correct way in applying the Chi-Square test.

• Are the high income subjects part of the 'so-called population' or disjoint from it? Nov 14, 2019 at 16:59
• it's a part of it. But on that matter, what if the high income subjects are disjoint from it? Because I am trying to see if there is a significant difference between high-income and non-high-income subjects. @BruceET Nov 14, 2019 at 17:05
• If high income are part of the larger sample, then you must adjust count in larger sample to exclude high income. Nov 14, 2019 at 17:10
• Before your last comment, I ran the chi-squared test assuming disjointness. Large P-value. No rejection. Nov 14, 2019 at 17:23
• Thanks for running that test with disjointed distribution! I would like to know though why we must disjoint? I understood that including the high-income sample as part of the population might affect something but I don't know exactly what it affects? Nov 14, 2019 at 17:26

According to your Comment, I'm subtracting the high-income subjects from the population before doing the chi-squared test in R:

TBL = rbind(c(260,480,50)-c(50, 95, 8), c(50, 95, 8))
chisq.test(TBL)

Pearson's Chi-squared test

data:  TBL
X-squared = 0.4215, df = 2, p-value = 0.81


Do not reject. Observed and expected counts match quite well:

chi.out =chisq.test(TBL)
chi.out$$obs [,1] [,2] [,3] [1,] 210 385 42 [2,] 50 95 8 chi.out$$exp
[,1]      [,2]      [,3]
[1,] 209.64557 387.03797 40.316456
[2,]  50.35443  92.96203  9.683544


So the Pearson residuals are all very small. The chi-squared statistic is the sum of squares of the Pearson residuals.

chi.out\$res
[,1]       [,2]       [,3]
[1,]  0.02447869 -0.1035910  0.2651450
[2,] -0.04994731  0.2113713 -0.5410126

• So what does it mean? Since we don't reject the null hypothesis, can I interpret it as high income level is not a good factor in determining whether somebody will exhibit fraudulent behavior? Nov 14, 2019 at 17:24
• You don't have much data. I would say you have no evidence that high income is related to fraudulent behavior. Perhaps with more data, some evidence of that might appear. Nov 14, 2019 at 17:27
• How much is much? I read somewhere that having too many data might not be good as the test will become very sensitive to a relatively minor variation (in percentage) in sample size difference but large percentage in terms of count/numbers. Nov 14, 2019 at 17:29
• With 'too much' data you might detect an effect that is too small to make a practical difference. But here you'd need more data to have a realistic chance of rejection. Nov 14, 2019 at 17:31
• Got it. Thanks for the explanation for now! :) This is almost about all the data I can get for now. Asking for larger sample size provides a problem of its own. So I have to make do with what I have right now. Nov 14, 2019 at 17:34