1
$\begingroup$

I am unsure about which statistical test suits my needs, in particular I wonder whether chi-square should be used or not. The responses I get from my experiments are not continuous, but discrete, specifically taking the values of 0 or 1. I need to compare 3 groups. Here the details:

I performed 3 experiments, each experiment tested a different system, and each experiment involved a distinct group of subjects. In total there where 25 participants (7 for the first system, 7 for the second, 11 for the third). Subjects were asked to identify a stimulus (i.e. a vibration provided by a haptic device).

Recognition was measured as 1 = correct identification, 0 = incorrect identification. Stimuli where repeated, each stimulus receive 0 or 1 as measurement of participants' responses to it.

The stimuli where different for each system. Still I want to test whether participants using one system performed better than the participants using other systems.

My goal is only to assess the statistical differences between the 3 groups.

Based on my understanding, an alternative to chi-square could also be a binomial logistic regression, but I am unsure. If chi-square is the right way to go, can I compare 3 groups using 3 separate tests (A vs B, B vs C, A vs C) without affecting the alpha? If I have to correct it, what is the right value for 3 groups?

Can anyone please suggest the right function to be used in R?

Improve this question
$\endgroup$
2
  • $\begingroup$ Where the stimuli repeated for each participant? How many times? $\endgroup$
    – kjetil b halvorsen ♦
    Commented Nov 11, 2018 at 15:41
  • $\begingroup$ yes there were 3 repetition per participant $\endgroup$
    – L_T
    Commented Nov 11, 2018 at 22:15

1 Answer 1

Reset to default
2
$\begingroup$

Chi-square have H0 that the distribution among all groups is the same. So it will be significant if just one group is different, but you won't know which.

Logistic regression will be more informative, since it will create a set of dummy variables for each group and run test whether coefficients for each dummy variable is statistically significantly differ from 0.

Some code

library(tidyverse)
data_frame(a = sample(c(0, 1), 20, prob = c(0.4, 0.6), replace = TRUE),
           b = sample(c(0, 1), 20, prob = c(0.6, 0.4), replace = TRUE),
           c = sample(c(0, 1), 20, prob = c(0.8, 0.2), replace = TRUE)) %>% 
   gather(var, value, a:c) ->
   df

fit <- glm(value ~ var, data = df, family = "binomial")
summary(fit)

df %>% 
  count(var, value) %>% 
  spread(value, n) ->
  df

chisq.test(df[,-1])

UPDATE

If you want to compare all three groups you could relevel the var variable, using factors.

df$var <- factor(df$var, levels = c("b", "c", "a")) # now b will be a reference point
fit_2 <- glm(value ~ var, data = df, family = "binomial")
summary(fit_2)
Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks a lot. I need some more help as I don't fully understand how to interpret the results: This is the formula I used fit <- glm(Response ~ System, data = scrd, family = "binomial") and then summary(fit). I got that the intercept and the first System lead to a significant p-value. However it is not clear to me what I can infer from these results. My goal is to decide if one system led to significantly better recognition performance compared to the other two. $\endgroup$
    – L_T
    Commented Nov 11, 2018 at 22:10
  • $\begingroup$ Call: glm(formula = value ~ var, family = "binomial", data = df) Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) 1.7346 0.6262 2.770 0.00561 ** varb -2.3536 0.7823 -3.009 0.00262 ** varc -3.9318 0.9735 -4.039 5.37e-05 *** This is a result from summary(fit). Since two dummy variables are created the intercept is a vara, and all other varb and varc are compared to vara. So the H0 for each coefficient is that the difference between it and reference level (vara in my example) is equal to zero. $\endgroup$
    – aGricolaMZ
    Commented Nov 11, 2018 at 22:59
  • $\begingroup$ Thanks a lot for your kind and timely answer. This however tells me that var a is different from varb and varc, but what about the difference between varb and varc? How can I get it? Moreover, how can I report the results of the analysis in a scientific paper? Thanks in advance $\endgroup$
    – L_T
    Commented Nov 11, 2018 at 23:10
  • $\begingroup$ Thanks for your question! Suddenly I realized that everything is not so clear. If you want to compare all groups there are two strategies: 1) Use pairwise comparisons and then adjust obtained p-values 2) Use contrasts. $\endgroup$
    – aGricolaMZ
    Commented Nov 12, 2018 at 8:32
  • $\begingroup$ Thanks Jerzi. I am using the wald test. What do you think? I have applied the test and the results are available here: stats.stackexchange.com/questions/376511/… If I want to use the contrasts, would you be so kind to provide some code in relation to your example below, as well as how to interpret the results? $\endgroup$
    – L_T
    Commented Nov 12, 2018 at 10:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.