Levels of variable and $\chi^2$

Question

Does it matter how many levels the independent variable has for significance? For example, suppose socioeconomic status has 10 levels and the output variable has 2 levels. Suppose this is significant in a $\chi^2$ test. If we increase the number of levels to 20 for the independent variable would this change the significance?

The reason I ask is because I can't seem to put my independent variable into 10 levels. Instead there are 30 levels (basically a listing of all the values). Would this matter for significance?

Answer 1

"Splintering" one or both variables does reduce statistical power--the probability of finding a result significant if it "deserves" to be. When you splinter a variable into many levels, you create a condition in which there are many, many ways in which disproportionality could arise just by chance. Thus a Chi-Square test would need to use more degrees of freedom, which requires a stronger disproportionality before a result would reach statistical significance.

See whether you can condense your 30 categories into a smaller number. If not, you may need to be extra "lenient" when it comes to your alpha level--that is, your requirement for statistical significance. For example, instead of setting alpha at .05, you might raise it to .10 or even .20, depending on the expectations of your audience and your/their feelings about the relative costs of Type I and Type II errors.