# T-Test using means vs without means - correct approach

The data are from book ratings done by users. Users were split into three experimental groups (A/B - test). Goal is to understand if users show any difference in rating books between these groups. Carrying out t-test sounds like the first approach to understand the differences.

Since there is slight imbalance in the user populations i.e: Experiment Group1 has 100 users, Group2 has 110 users, Group3 has 105 Two Sample t-test seems sensible.

1. Executing two sample t-test between Group1 Vs Group2, Group1 Vs Group3, and Group2 Vs Group3 with-out calculating average book rating per user.

Here taking each rating done by each user and running t-test. Exp:

import scipy.stats as stats
group1Rating = [2.5, 2, 4.5, 4, 4.5, 3.5.....4] #multiple rating per user
group2Rating = [3, 2, 5, 4, 4, 4.....4.5] #multiple rating per user
grp1-grp2 = stats.ttest_ind(a=group1Rating, b=group2Rating, equal_var=False)

...


Here I see p-values 3.24 * 10^-12

similarly between rest of the group combinations.

2. Executing two sample t-test between Group1 Vs Group2, Group1 Vs Group3, and Group2 Vs Group3 with calculating average book rating per user (Each user has a single record which is average rating).

Here I see no significance between the p-values Example: Ttest_indResult(statistic=0.984, pvalue=0.324)

Questions:

• Which one is the correct approach? If none of the above whats the correct t-test for above scenario.
• Is there a different/alternate test to understand the rating differences between groups.