# Sample size calculation for paired ordinal data

What formula should be used for sample size calculations for paired ordinal data? Imagine we have an intervention and we want to measure the effect based on before and after measurements of an ordered outcome (say a Likert scale).

I know how to calculate for a t-test for paired continuous data and how to calculate for a Mann-Whitney test for ordinal data (Whitehead 1993), but what about paired ordinal?

Either a nonparametric solution or a parametric solution through a GLM for ordinal response would work I think.

• You may find the following paper helpful Mike Julious SA, Campbell MJ, and Altman DG (1999). Estimating sample sizes for continuous, binary, and ordinal outcomes in paired comparisons Journal of Biopharmaceutical Statistics, 9, 241-251 Jun 23, 2015 at 11:05

The standard non-parametric test for paired ordinal data is the Wilcoxon, which is sort of an augmented sign test. I don't know of a formula for power analysis for the Wilcoxon, but you can certainly get power analyses for the sign test (there are various resources listed in my question here: Free or downloadable resources for sample size calculations). Note that (as @Glen_b notes in the comment below), this would assume that there are no ties. If you expect there will be some proportion of ties, the power analysis for the sign test would give you the requisite $N$ excluding the ties, so you would inflate that estimate by multiplying it by the reciprocal of the proportion of untied data you expect to have (e.g., if you thought you might have $20\%$ tied data, and the test required $N=100$, then you'd multiply $100$ by $1/.8$ to get $125$). Unless you need the minimum $N$ to achieve a specified power, that should work for you. For example, when running power calculations for more complicated analyses, we often use a simpler calculation and then say something like 'our $N$ was calculated to achieve a minimum of 80% power on the sign test, because the Wilcoxon can be expected to be at least as powerful as the sign test, our power should meet or excede 80%'.

On the other hand, if you have a strong sense of what the distributions will be like, you can always simulate. Although written in the context of logistic regression, there is a lot of basic information about using simulations for power analyses in my answer here: Simulation of logistic regression power analysis designed experiments.

• There are some power analyses for the signed rank test, but under the assumptions of the test (i.e. continuous, not ordinal data). Much the same issues applies for the sign test - the calculations assume you're not dealing with some proportion of ties (pair-differences of zero). The way to approach power (and significance level) for the signed rank test on ordinal data would be to simulate from whatever set of circumstances are relevant -- as gung suggests in the second paragraph. Aug 24, 2014 at 4:22
• @Glen_b, good point about the ties, I edited the answer to reflect that. I also recall (now that you mention it) that there are formulas for power analysis when the data are normal but the things I've seen always seemed too ad-hoc for my taste (eg a quickie approach is to inflate the N for the t-test by the reciprocal of the ARE). Aug 24, 2014 at 12:53
• In anything but quite small samples, the "quickie approach" based on the ARE works surprisingly well for normal data, if you adjust to use the actual significance level you will have with the signed rank test (rather than the nominal). Here is signed rank power (via simulation from normal samples) at n=16 (points), and the implied power curve for a t-test with variance adjusted to compensate for the ARE (red curve), using the attainable significance level. It's not so useful on non-normal data of course. Aug 25, 2014 at 0:45
• See also the discussion/reference in this post Aug 26, 2014 at 13:40