I've run a survey which received around 270 responses from a population of around 5,500. The survey asks brief demographic questions followed by opinion questions, akin to the following:

  • Demog Q 1: Do you work in an urban or rural setting? (A or B)
  • Demog Q 2: Are you employed or self-employed? (A or B)
  • Opinion Qs: On a 5-point scale from strongly disagree to strongly agree, please rate the following:
    1. Red is the best colour.
    2. Blue is the best colour.
    3. Yellow is the best colour.
    4. Black is the best colour.

(I can't reveal the actual questions since this is unpublished work and the sponsor may object to the questionnaire being made public before publication.)

Overall there was no clear preference for any single colour, so I wanted to drill down by demographic. My initial null hypothesis was that there would be no difference, that demographics do not influence colour preference.

To test the null hypothesis, I sub-grouped responses by demographics and conducted Kruskal-Wallis H test (a.k.a. one-way ANOVA on ranks) on each demographic sub-group with alpha at 0.05. I found the null hypothesis rejected for all survey questions among all sub-groups, except one -- let's say question 4 about black.

I'm interpreting this to indicate that preference for black is unrelated to the demographics collected, whereas preference for red, blue or yellow are all related to demographics.

If, then, I was to launch a series of red, blue, yellow and black products into the market, the demographic factors I've collected (urban/employment) may be considered irrelevant in marketing campaigns for the black products.

Is this a reasonable interpretation of the KW result? I appreciate this may be a matter of opinion, but I would like to understand if I'm using the test appropriately.

Should a Dunn test be conducted, and what further information would this show?

  • 1
    $\begingroup$ It's a little unclear what you have done. For example, you say "I sub-grouped responses by demographics". But you have two demographic questions. Did you treat these separately? Or did you combine the two demographic questions? Second, I assume you are running a separate KW analysis for each of your color questions. So how many tests have you done it total? $\endgroup$ Oct 19, 2019 at 15:32
  • 1
    $\begingroup$ To start to form an answer: The KW approach is generally appropriate here. Be sure your implementation of the test accounts for ties. The Dunn test (1964) will give you additional information only if there are more than two groups in the independent variable of the KW test. A more sophisticated & flexible approach would be ordinal regression, and that would allow you to examine the interaction of Urban and Self-employed (which may be really valuable here). $\endgroup$ Oct 19, 2019 at 15:35
  • 1
    $\begingroup$ Finally, 270 is relatively large sample size. Don't over-rely on p values. Be sure to look at some kind of effect size. This could be simply presenting a bar plot (like a histogram) of frequencies for each group. Or differences in medians. Or effect size statistics like epsilon-squared or Vargha and Delaney's A between groups. $\endgroup$ Oct 19, 2019 at 15:37


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.