Factorial ANOVA with composite dependent variable and interaction

Question

I ran an experiment where participants were randomly assigned to a control or treatment condition and in both conditions each participant was presented consecutively with three pairs of products where one product was more hedonic and the other more utilitarian. They had to choose one product in each of the three choice sets (1=hedonic, 0=utilitarian), that is they had to make three choices in total. Every choice set was a different product category.

I combined the three binary answers from every participant to form a "hedonic choice index" and I would like to examine the effect of the condition on this choice index. However, I would also like to check if the effect differs for the product categories.

I was wondering if the right way to do it is to run a factorial ANOVA like this:

df <- df %>% mutate(across(c(1,2,3,4,5), as.factor))

library(rstatix)
anova_test(data = df, choice_index~condition*product_category1+condition*product_category2+condition*product_category3, effect.size = "pes")

Here is my data:

 df <- structure(list(respondent_id = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20), 
                 condition = c("control", "treatment", "control", "treatment", "control", "treatment", "control", "treatment", "control", "treatment", "control", "treatment", 
                               "control", "treatment", "control", "treatment", "control", "treatment", "control", "treatment"), 
                 product_category1 = c(0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0), 
                 product_category2 = c(1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0), 
                 product_category3 = c(0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0),
                 choice_index = c(1,2, 0, 3, 1, 3, 1, 3, 3, 0, 1, 2, 0, 3, 1, 3, 1, 3, 3, 0)), 
            class = "data.frame", row.names = c(NA, -20L))

Answer 1

Your data is fundamenaelly categorical (picked utilitarian vs. hedonistic product). You coded the results as (0,1), but that does not mean that you can do any arithmetic on this coding; you could as well have coded the results (0,100), or A/B, or happy face/sad face. Any arithmetic operation on these values is mathematical unsound; that means sums, averages, standard deviation, etc... It is a common mistake (trap?), so do not feel bad, but ...
If you think about it, when subject 3 picked 1 over 0 for the first product, he did it w/o any hesitation, by a large margin. But when subject 4 did the same, she barely perceived a difference, and picked one only because she only had these 2 choices. But you gave both the same score. Any arithmetoic result you get from that is meaningless. All you can say is that both subjects preferred 1 to 0.
You should use a binomial test: for each product you count how many picked 0, and how many picked 1, and then you use a binomial test. In this case a 2x3 contingency table (2 groups -control & treatment- accross 3 categories -3 products-). You can for example use this web page, which has such a calculator (but most software will also have functions to support these tables).
Since your counts are quite low, only the Fisher-exact method is valid (normal approximations, aka $\chi^2$, are not appropriate).
Fwiw, based on your data, for the overall comparison (is there a difference between the choices of the 2 groups?), the answer is No (p-value=.32. Btw, $\chi^2$ gave p=.34, and likelihood ratio gave .33). So all the methods agree. Now, if you only test for product 1, there is a significant difference (indeed, control picked hedonistic only 2 out of 10 times, while treatment picked it 8 out of 10 times). The p-value is .023. BUT... If you also do the other 2 comparisons (for the other 2 products), which are both non-significant, then you should use a multiple comparison correction. And with e.g. Bonferroni, comparing to 0.017, then even for product 1, the difference is not significant. So if you look at an overall test, or at a set of 3 comparisons, you can not find a significant result.