1
$\begingroup$

I am currently in the design phase for an experiment on different packaging attributes and their effects on an actual purchasing choice. In this context, I am asking myself a possibly trivial question: Is the follwing design a true within-design or not?

Each participant chooses one out of 16 products (presented at the same time), which represent the combinations of 4 independent variables (IVs) with two levels each:

  1. size: large or small
  2. shape: bottle or box
  3. material: recycled or not
  4. brand: sustainable or not

In one sense, it is a 2x2x2x2 factorial design with the resulting 16 conditions/combinations.

I have understood that a within-design means, that each participant is tested under all conditions. Usually, this is done sequentially - in my research, I would present each participant with a choice between large or small, and then a choice between bottle or box, and then... etc. However, what if I present the participants with a choice out of all the 16 combinations at once? (as described above)
Is this still a true within-design?

And what does it tell me regarding sample size and analysis tools?
Do I need

  • only 30 participants since each makes only one choice (which tells me a combination of the 4 IV they prefer)
  • or do I need 30x4=120 participants (since I have four IVs)
  • or do I actually need 30x16=480 participants (because of the 16 conditions/combinations)?

In this sense, I am also trying to figure out the sample size needed dependent on how many packaging attributes are included as independent variables and whether the experiment is set-up as within or between-subject design.

I have browsed literature on experimental design and power analysis, but haven't come to a clear conclusion yet. Hence, I would be very grateful for some perspective on this.

Improve this question
$\endgroup$
5
  • $\begingroup$ A couple of comments. Getting someone's opinion is highly subjective thus more samples the better. Also asking an opinion for one out of 16 choices is probably unproductive. I would consider either reducing the number of options by running a half or maybe even a 1/4 factorial experiment. The interaction term between 4 or 3 terms is unlikely to be significant. Or another option is to run subsets of the 16 products with a balance incomplete block design. $\endgroup$
    – Dave2e
    Commented Apr 14, 2020 at 17:37
  • 2
    $\begingroup$ See this paper on sample size calculations for discrete choice experiments, which is the kind of study you have here. Any literature on ANOVA (presumably where you got the sample size of 30) is irrelevant here. $\endgroup$
    – Noah
    Commented Apr 14, 2020 at 19:17
  • $\begingroup$ Thank you for sharing your thoughts! @Dave2e Could you please elaborate what you mean with 1/4 factorial experiment? I understand the balance incomplete block design to go into the direction of sequentially presenting the subjects with choices of just one independent variable, and randomize their order, correct? Initially, we thougth 16 choices is quite realistic for a normal supermarket visit when it comes e.g. to washing detergent, but I totally see your point that in this design, interactions btw. 4 independent variables will be hard to filter out... $\endgroup$
    – kborca518
    Commented Apr 15, 2020 at 7:47
  • $\begingroup$ Thank you @Noah for helping me categorize this kind of design as discrete choice experiment - I am looking into your recommended read now. $\endgroup$
    – kborca518
    Commented Apr 15, 2020 at 7:53
  • $\begingroup$ @kborca518, I miss spoke earlier, with 4 factors the 1/4 factorial is not an option. Your options are either a full factorial with 16 samples or a half factorial with 8 samples. Concerning your question: "presenting the subjects with choices of just one independent variable, and randomize their order, correct?" What I meant here is present your subjects a subset of samples (randomize 2-4 samples) and have them rank/grade their preference. There is a area of experiment design related to this type of survey, sorry I can't find a good reference to it now. $\endgroup$
    – Dave2e
    Commented Apr 15, 2020 at 12:20

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.