0
$\begingroup$

I'm trying to figure out if there's a difference across the main capitals in Europe and the voting preference of their inhabitants for extreme right political parties. So far I have categorized all people who voted into either extreme right (=1) or not (=0). That gives me a table with the capitals as rows, and one column that shows the number of people who voted for an extreme right party and one column for all people who did not. So far so good.

The next step would be to do a chi squared test to check if there is a significant difference in terms of distribution across the two groups (extreme right or not). The problem I have, is that the chi squared test is inappropriate for "large numbers". So I can sample my data by taking at random 5% of my data set and use that subset for my chi squared test, and repeat that 10 times to be sure that I'm getting the same outcome. But that seems not very elegant to me. The real question is now of course, how large does my data set have to be? And how much is too much data? I've Googled around a bit but I can't seem to get the right information. Any ideas?

$\endgroup$
  • $\begingroup$ Perhaps you should post the reference that has told you that chi-squared tests are problematic with large samples, as my guess is that the problem it is referring to is a bit different to the one you have described. $\endgroup$ – Tim May 23 '16 at 7:22
0
$\begingroup$

The chi-squared test is awesome with big sample sizes. In fact, the bigger the better. So, you have no problem at all.

$\endgroup$
  • $\begingroup$ In large samples The $\chi^2$ test will give correct $p$-values, but those will be meaningless. You will find statistically significant differences between the capitals, but are they substantively significant. The solution is not to sample a smaller sub-sample, but instead to take the entire sample and just look at the estimates and decide whehter or not you find those differences big. $\endgroup$ – Maarten Buis May 23 '16 at 8:08
  • $\begingroup$ You can have a huge sample and still get no significant difference, and you would want to know that IMO. And, in most contexts, reporting an effect size without a p-value will win few friends. Having said that, it all comes down to what @Bram Van Camp means by large. If he is talking about one of the large social surveys, the tests will be useful. If he is looking at voting records, they will not. $\endgroup$ – Tim May 23 '16 at 8:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.