I am running a MaxDiff experiment where there will be four different "arms". Essentially, a respondent will be assigned to one of four conditions, where they will be shown one of three messages (plus a control), and then they will take the MaxDiff battery (which will include 20 items).
My question has multiple components. I know how to create a MaxDiff design (without covariates) in R
using choiceDes
or AlgDesign
. But I do not know how to include the arm of the experiment (which condition the respondent is assigned to) as a covariate in generating the design.
I could, however, assume that I replicated the design across all four arms. But I would still be stuck here because I don't know how to evaluate the power of the design (whether or not covariates are included)
So I don't know how to:
- Generate a MaxDiff design with covariates (in this case, the arm of the experiment)
- Evaluate the power of the design (conditional on sample size) of a MaxDiff experiment with covariates
- I also don't even know how to evaluate the power of the design without covariates
The goal here is to know what the sample size must be to in order to detect changes in utilities for the attributes across the different arms. We can of course make assumptions as needed about the effect size, etc.
The package skpr
seemed promising. I had thought that I could generate a MaxDiff design (e.g. using choiceDes
), assume that I replicated the design across all four arms, and use eval_design_mc
to evaluate the power of the design. However, this only takes gaussian, binomial, poisson, or exponential families as distributions, and the actual model here is multinomial, not binomial.
So... I'm a bit stuck.
If necessary, I could simulate a dataset (making assumptions about effect sizes and sample size) and estimate the model using simulated data using mlogit
, and use the resulting standard errors as a guide, but I'm hoping that there is a simple rule of thumb.
I have been through all of the documentation and rules of thumb that I can find online, e.g. this comment which talks about rules of thumb for subgroup analysis (but does not give any indication why or how to assess the quality of the experiment) and the linked paper ("What is the right sample size for my MaxDiff study?" in this pdf, which only covers the case without subgroups/covariates).
I am hoping there is a (relatively) easy way of thinking about this that does not require simulating a dataset. Any help would be greatly appreciated!