# Analysing categorical variables in R with chisq.test, with small/no expected values

I was wondering if anyone could help me with analysing data in R.

I am working with transplant data, I am looking to compare outcomes between on/off bypass at surgery. I previously asked for help analysing my categorical variables here: https://stackoverflow.com/questions/37094104/apply-analysis-on-multiple-columns-in-a-dataset-split-by-factor-including-conti

I have used the code provided in the above link to test my categorical variables, for example, to look for a significant difference between the number of Males in group 1 and 2 (see example data below). However, I have come across two issues when analysing the categorical variables.

When running the test I have found that if the groups are small I receive this message

"Warning message:
In chisq.test(tab$n) : Chi-squared approximation may be incorrect"  Here is some example data: library(dplyr) df <- read.table(text="Group,Sex 1,Male 1,Male 0,Male 0,Male 1,Male 1,Male", header = TRUE, sep = ",") # tally number of participants in each Group by Sex tab <- tally(group_by(df, Group, Sex)) tab chisq.test(tab$n)  # test for Group differences by Sex


The output:

> tab
Source: local data frame [2 x 3]
Groups: Group [?]

Group    Sex     n
(int) (fctr) (int)
1     0   Male     2
2     1   Male     4

> chisq.test(tab$n) # test for Group differences by Sex Chi-squared test for given probabilities data: tab$n
X-squared = 0.66667, df = 1, p-value = 0.4142

Warning message:
In chisq.test(tab$n) : Chi-squared approximation may be incorrect  I have had a look online and have read that this is because the expected values are very small, and one solution is to use: chisq.test(tab$n, simulate.p.value = TRUE)


I was wondering if this was the correct thing to do, and whether there would be any advantage to using Fisher's exact test. If Fisher's test is the preferred method I would probably need some help running this on R.

The second issue I have is when some variables are only present in one group, when I test these I get the following message:

Error in chisq.test(tab2$n) : 'x' must at least have 2 elements  Example data: library(dplyr) df2 <- read.table(text="Group,Sex 1,Male 1,Male 1,Male 1,Male 1,Male 1,Male", header = TRUE, sep = ",") # tally number of participants in each Group by Sex tab2 <- tally(group_by(df2, Group, Sex)) tab2 chisq.test(tab2$n)  # test for Group differences by Sex


What is the best way to test if the number of Males in group 1 is significantly greater than group 2, which is 0?

# Later edits copied from another version of the same question

EDIT:

Thank you for your reply, I had originally asked this question on StackOverflow hense the lean towards R. I may need help with the R side of things too, however any input on the type of tests to use would be appreciated.

My data includes two groups, 'Bypass on' and 'Bypass off'. Although you have asked for a more complex dataset the one I provide is representative of what I am working with.

I am looking to see if there is a difference between 'Bypass on' and 'Bypass off' across a number of variables, including age, height, weight, sex.

I have been using a chisq.test in R to look at the categorical variables, in my example there are 4 males in the 'Bypass on' group and 2 males in the 'Bypass off group'. I want to test whether there is a significant difference between the number of males in each group, however I am getting the following message from R (which I believe is due to the small sample):

library(dplyr)
Bypass on,Male
Bypass on,Male
Bypass off,Male
Bypass off,Male
Bypass on,Male
Bypass on,Male", header = TRUE, sep = ",")
# tally number of participants in each Group by Sex
tab <- tally(group_by(df2, Group, Sex))

chisq.test(tab$n) # test for Group differences by Sex "Warning message: In chisq.test(tab$n) : Chi-squared approximation may be incorrect"


I was wondering if Fisher's exact test would be suitable in this situation, as opposed to using:

chisq.test(tab$n, simulate.p.value = TRUE)  The second problem I have is when there are 6 males in the 'Bypass on' group and 0 males in the 'Bypass off' group. When I use the chi.square test on R I receive the following message: Error in chisq.test(tab2$n) : 'x' must at least have 2 elements


I am not sure what test to use in this situation.

EDIT2:

I will try to explain what I am actually trying to do in the big picture...

My data includes a number of postoperative outcomes such as, blood loss and hospital stay. These are the variables I am really interested in, I am trying to find out if there is a difference in these postoperative outcomes between patients who had 'Bypass on' and 'Bypass off' at surgery.

I am firstly running analysis on some of the baseline variables to ensure they are not effecting the analysis, to put it simply I want to be able to say Sex isn't going to effect analysis of postoperative outcomes as there was no significant difference between the number of males/females in the groups.

Here is an example of the dataset:

df <- read.table(text="Group,Age,Sex,Diagnosis,Blood loss,Hospital stay
On bypass,25,Female,Diagnosis 2,444,21
On bypass,40,Female,Diagnosis 3,272,13
On bypass,20,Male,Diagnosis 2,325,12
On bypass,35,Male,Diagnosis 1,444,11
Off bypass,32,Female,Diagnosis 2,277,11
Off bypass,25,Female,Diagnosis 2,496,18
Off bypass,40,Female,Diagnosis 1,396,18
On bypass,31,Male,Diagnosis 2,416,14
On bypass,22,Female,Diagnosis 4,372,7
Off bypass,23,Female,Diagnosis 4,203,22
Off bypass,25,Male,Diagnosis 1,313,16
Off bypass,24,Female,Diagnosis 3,420,23
On bypass,36,Male,Diagnosis 2,387,18
On bypass,36,Male,Diagnosis 4,341,21
On bypass,21,Female,Diagnosis 1,498,23
On bypass,27,Female,Diagnosis 3,462,13
Off bypass,20,Female,Diagnosis 2,243,15
Off bypass,30,Female,Diagnosis 4,330,8
Off bypass,30,Female,Diagnosis 3,372,22
On bypass,36,Male,Diagnosis 4,367,11
On bypass,23,Male,Diagnosis 3,201,9
Off bypass,23,Female,Diagnosis 4,225,5
Off bypass,31,Female,Diagnosis 3,323,14
Off bypass,27,Female,Diagnosis 4,495,8", header = TRUE, sep = ",")


As you can see there are females in my data, however I split these to analyse (as I did with the other categorical variables). The reason I split Sex is because I want to test whether both groups have a similar amount of males, and then run a separate test to see if both groups have a similar amount of females.

I may be over complicating things, hopefully that makes sense.

• This is not really a coding question. Although, you can use fisher.test perhaps. May 11, 2016 at 19:50
• Flagged to migrate to CV... I suggest you do use Fisher's exact test. May 11, 2016 at 19:53
• If you need help choosing an appropriate statistical analysis, you should post your question to Cross Validated, not here. May 11, 2016 at 20:25
• Thank you for the replies, I will post this question to CV. Thanks, Tom May 11, 2016 at 20:42
• (Note that asking just how to do something in R is generally off topic here. I think there are statistical confusions we could clear up & so this could probably stay open, but you may want to edit to emphasize the statistical issues & deemphasize the "help me with... in R" aspects.) May 11, 2016 at 20:51

In your example you have only males and your use of chisq.test is testing goodness of fit. You can specify the hypothesised proportions of each group but if you do not then the default is for them to be equal. The advice to use fisher.test instead is only useful if you have a contingency table. If you are concerned about the small frequencies then since you only have two outcomes you can use .

But is this really what your scientific/clinical question is? Do you really want to test whether there are equal numbers of group 0 and group 1 amongst the men?

As far as I can tell from your question, you could use a goodness of fit test. In this case, you would be testing the hypothesis that the proportion in Group 1 is 50% of observations. For two groups and an "expected" or "default" proportion of 0.5, for your final example with six males:

x = 6
n = 6
expected = 0.5
binom.test(x, n, expected)


If there are six observations with two in Group 1:

x = 2
n = 6
expected = 0.5
binom.test(x, n, expected)


I suspect the small counts don't bother the binom.test function.

Source

If the number of males and females are roughly equivalent in each group, then you might want to investigate sex * group interactions.

Aside from this being in the wrong place. The first warning is about sample size, you should do a power analysis to see what would be the minimum number of samples needed to use the chisq and be sure to have that minimum if you intend to make significant inferences.

In the second analysis warning, the x warning is telling you that you have a only one observable valuable in the group column, all of the values are one and in the sex column all of the variable values are male, again 100%.

There is no way to compare values when there is no variation in the values.