Difference of output in Chi-square in SAS and R

Question

Currently I have a dataset with an N of 3866 with two variables, Project (group variable, six levels) and Topbox (two levels; yes, no). We are looking to see if there is a difference in the proportion of 'Yes' responses between different projects (i.e., is there a significant difference between the project that has the lowest proportion of 'yes' responses versus the project that has the highest proportion of 'yes' responses). I've conducted a Chi-Square test in both SAS and R, and both give me different X^2 and p-values. Outputs shown below:

SAS:

proc freq data = aggregate; 
tables q57_topbox*project / chisq;
run;

Frequency output:

Chi-square output:

R:

chisq.test(records_filter$q57_topbox, records_filter$project, 
           correct=FALSE)

Chi-square output:

    Pearson's Chi-squared test

data:  records_filter$q57_topbox and records_filter$project
X-squared = 21.216, df = 5, p-value = 0.0007373

I have a couple questions regarding this.

1. The X^2 and the p-value are different and I am not sure why.
2. Looking more into this, I'm not sure if a Chi-square statistic is the right test for such a dataset. The cell counts are not only widely variable (counts between 37 and 1143 in the 'Yes' response), but the cell counts are also bigger than what I'm seeing most people would use a Chi-square for. Therefore I'm unsure if this is the correct test to begin with.

Any help at all is appreciated, thank you so much for reading. Please let me know if there's any other info I can provide (I'm quite new to asking questions to stack exchange, so apologies!)

Answer 1

The difference comes from SAS using "NA" as own category. If this is what you are after, then you can replace the NA in R by some explicit missing like "missing".

Regarding your second question: You could use an extension of Fisher's exact test for general two-way tables instead. But in practice, such considerations are often less important than they seem to be. Much more important are questions like "is my sample really randomly picked or is there some bias?" or "which hypothesis would I want to show?"

Answer 2

One thing is that the SAS program is including the NA (hence, df = 10) while R is not (hence, df = 5). As the proportion of NA varies hugely by project, SAS is given a much higher $\chi ^2$ and a much lower p.

I think that you probably want to exclude the NA, but it's really dependent on exactly what NA means, how it arose, and what your overall purpose is.

As for using chi-square: Neither the variation in cell sizes nor the cell sizes themselves are reasons not to use it. The real question is whether your question would be better served by a regression than by chi-square. There are a couple differences. One is that regression marks one variable as "depenendent" or "response" and the others as "independent" or "predictors". Chi-square treats the variables interchangeably.