Understanding hypothesis testing, confidence intervals and frequencies

Question

Sorry if this is a silly question but I seem to be mixing a few things together and would love if someone could straighten it all out for me. Please correct me!

This is somewhat related to my post here: Proportion Composite score for DV in rANOVA- Need Help Interpreting Results

I conducted a study where all participants answered 3 binary items in each of 4 conditions (2x2 design where each fixed, within subjects factor was crossed with the other). I am not including any between subject factors . The 3 items were averaged to form a proportion (yes, yes, I have seen the Dixon and Jaeger articles but I don't want to get into multi-level modeling so bear with me, please. If you do think I should do GEE- please tell me how since I have never used it before).

My first hypothesis is that in the 1,1 and 1,2 conditions, people will be more likely to choose the less expensive product (A in 1,1 and B in 1,2). My second hypothesis is that in both 2,1 and 2,2 people will be more likely to choose B.

I ran a repeated measures ANOVA and saw that the 2 main effects and the interaction are significant. I also got a table with the means and confidence intervals.

1. Instead of running an analysis, should I have compared each level separately to the criterion 0.5? Which test should I use since the data across conditions is correlated?

2. How do I explain main effects and levels in the interaction? Do I compare the levels to each other (i.e. are they different) and then look at the CI to see which direction each one is going (i.e. > or < than 0.5).

3. How do I interpret a CI that contains 0.5? Can I say that that level does not the preference for a product?

I feel like I have a lot of things swimming around here. I appreciate all the help so thanks in advance.

Answer 1

(1) If determining whether or not the preference is above 0.5 is important to you then I think comparing each to 0.5 is appropriate as long as you adjust for making multiple comparisons. For (2) you can do this by one-sided hypothesis testing but again need to account for doing multiple comparisons. For (3) my answer is the same as what I gave to you in your earlier post. The CI containing 0.5 indicates that there was not an overwhelming preference to the product.