I'm analyzing a survey over time, last year and this year, and I've been asked to compare the proportion of the two time periods for significance, from the same survey and same question data is categorical and there are three options; just different years. I am unsure as to which test I should run. I was thinking chi-square? But I also know chi-square requires independence, I'm unsure if that assumption is satisfied. If independence (is)n't satisfied which test should I use? Thank you, in advance.

  • $\begingroup$ Try the logistic regression with random effect. $\endgroup$
    – user158565
    Jul 24 '19 at 15:42
  • $\begingroup$ I am not sure I understand exactly what you are doing. My interpretation is shown in my Answer below. If my interpretation is right, maybe the answer will help. If it is wrong, please edit your question with appropriate clarifications, so one of us might be able to give a more helpful answer. $\endgroup$
    – BruceET
    Jul 24 '19 at 19:36

Suppose 500 randomly chosen people took the survey last year. For the target question, counts for responses $1, 2, 3,$ were $a=(166, 171, 163),$ respectively.

Similarly, suppose 600 (different) randomly chosen people took the survey this year with respective counts $b=(116, 244, 240).$

If you want to know whether the pattern of responses differed between the two years, then you can make a contingency table DTA of the counts, and do a chi-squared test. Results from R:

       a   b
[1,] 166 116
[2,] 171 244
[3,] 163 240


        Pearson's Chi-squared test

data:  DTA
X-squared = 27.555, df = 2, p-value = 1.039e-06

Because the P-value is below $0.05$ the null hypothesis that responses are independent of year is rejected at the 5% level.

This is not surprising because I sampled last year's data with equally likely responses 1, 2, 3; but this year's data with the respective proportions: 0.2, 0.4, 0.4.

Note: Complete R code for my fake data and analysis is shown below.

set,seed(724)  # for reproducibility
last = sample(1:3, 500, rep=T)
a = tabulate(last)
this = sample(1:3, 600, rep=T, pr=c(.2,.4,.4))
b = tabulate(this)
DTA = cbind(a,b)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.