Concrete scenario: A random sample of adults in Idaho rate each of 'kombucha', 'apple juice' and 'grape juice' on a scale from 0 to 1. Does the population of Idaho assign significantly different mean ratings to these items?
Characteristics of the data (as I understand them):
- The samples are paired, not independent (an individual's rating of apple juice and grape juice may be correlated)
- The variance of the samples is not necessarily equal (kombucha may be more polarizing than apple juice)
- The distributions aren't necessarily normal (kombucha ratings may have a bimodal distribution, for example)
- The ratings are continuous
Tests that I've looked at:
- Friedman's test ~ Sounds like it would work, but the ranking step throws away information
- Welch's t-test ~ Seems to assume that the samples are independent
- Wilcoxon signed-rank test ~ Assumes that the sample variances are equal
TL;DR: What's a good paired difference test for a single sample where each entity produces two (or more) differently distributed continuous measures?