We are currently planning an experiment. We plan with the following variables:
- An independent variable "treatment" with four values (a, b, c, d)
- A number of further independent variables for confounding factors (mostly metric or dichotomous).
- Two dependent variables "time" and "correctness", both ratio scale
I'm currently looking into how to analyze the data once we have it. What complicates things (at least for me) is that there is a partial order relation (https://en.wikipedia.org/wiki/Partially_ordered_set) defined among the values of "treatment". In other words, "treatment" is neither fully ordinal, nor fully nominal: a < b < c and a < b < d, but c and d are incomparable (i.e. the theory does not say anything about their effect relative to each other).
Our hypothesis is that for treatments x and y, $x < y \Rightarrow time(x) < time(y)$ and $x \leq y \Rightarrow correctness(x) \leq correctness(y)$
So far, I came up with two ideas to handle the data:
- Treat "treatment" as nominal and do a MANCOVA. Then manually look at the effect sizes and directions.
- Forget about all the other independent variables and just calculate/test Spearman's rank-order correlation for three cases: c < d, c > d and c = d
Question: What are the pros and cons of the approaches? Are there other/better possibilities?
P.S.: To (perhaps) complicate things further: We plan to measure two (randomly chosen) treatments per subject, to gather more data and lower the effect of inter-subject variance.