The data are from book ratings done by users. Users were split into three experimental groups
(A/B - test). Goal is to understand if users show any difference in rating books between these groups. Carrying out t-test sounds like the first approach to understand the differences.
Since there is slight imbalance in the user populations i.e: Experiment
Two Sample t-test seems sensible.
Executing two sample t-test between Group1 Vs Group2, Group1 Vs Group3, and Group2 Vs Group3 with-out calculating average book rating per user.
Here taking each rating done by each user and running t-test. Exp:
import scipy.stats as stats group1Rating = [2.5, 2, 4.5, 4, 4.5, 3.5.....4] #multiple rating per user group2Rating = [3, 2, 5, 4, 4, 4.....4.5] #multiple rating per user grp1-grp2 = stats.ttest_ind(a=group1Rating, b=group2Rating, equal_var=False) ...
Here I see
3.24 * 10^-12
similarly between rest of the group combinations.
Executing two sample t-test between Group1 Vs Group2, Group1 Vs Group3, and Group2 Vs Group3 with calculating average book rating per user (Each user has a single record which is average rating).
Here I see no significance between the p-values
- Which one is the correct approach? If none of the above whats the correct t-test for above scenario.
- Is there a different/alternate test to understand the rating differences between groups.