# Which hypothesis test should I use for this given situation?

Let's say I have lot of users who spend money on various items. Considering Item A: For this item, I want to find whether there is a statistical difference in average spending in the month of October and September.

I am confused between ANOVA and Two Sample T-Test. I read somewhere that if you have one categorical variable and one continuous variable use T-test. Here, in this situation since I am only considering two months can I fit my situation in this case.

Whereas back of my mind, the month is a categorical variable with more than 2 categories. It was suggested to use the ANOVA test where you have one continuous variable and one categorical variable with more than 2 categories.

Please suggest the best hypothesis test which I should use in my scenario.

Mean spending in October: 4.35
Std Dev of October: 1.95
Var of October: 3.81
Number of records october: 395

Mean spending in Nov: 4.49
Std Dev of Nov: 1.96
Var of Nov: 3.85
Number of records Nov: 383

• Can you match spending, customer-by-customer, to get paired data between Oct and Sept? Oct 26, 2020 at 1:08
• @BruceET yea essentially I can bt that will make this complicated. Wouldn't it? Oct 26, 2020 at 1:10
• Please take a look at my answer, which attempts to convince you that the complication of looking at paired data could be worth the trouble. Oct 26, 2020 at 2:16

I put your summary statistics into a recent release of Minitab to do a 2-sample t test, with the result that there is no significant difference between the two months.

Two-Sample T-Test and CI

Sample    N  Mean  StDev  SE Mean
1       395  4.35   1.95    0.098
2       386  4.49   1.96     0.10

Difference = μ (1) - μ (2)
Estimate for difference:  -0.140
95% upper bound for difference:  0.090
T-Test of difference = 0 (vs <):
T-Value = -1.00  P-Value = 0.159  DF = 778


I have two reservations. (1) If you were able to find something like 350 customers, so that we can see customer-by-customer how many spent more in Nov then Oct, then we might have sufficient power to find a significant difference---provided that one exists. (2) Even with such large sample sizes, I would worry about using a t test if expenditures are really far from normally distributed. However, without normal data there are nonparametric paired tests that might be appropriate.

Maybe I can illustrate the importance of pairing with some fake data, using procedures in R:

Paired test: Shows highly significant difference.

x1 = rnorm(350, 4.35, 2)
x2 = x1 + rnorm(350, .12, .5)
t.test(x1, x2, pair=T)

Paired t-test

data:  x1 and x2
t = -5.219, df = 349, p-value = 3.094e-07
alternative hypothesis:
true difference in means is not equal to 0
95 percent confidence interval:
-0.18519286 -0.08381676
sample estimates:
mean of the differences
-0.1345048


Here are summary of the paired differences and a plot of them. For my fake data, a convincing majority of subjects increased spending in the second month. Your data may not show such a significant result with paired data, but we will never know without the 'complication' of looking at pairs.

d = x2 - x1
summary(d);  sd(d)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
-1.3067 -0.1700  0.1231  0.1345  0.4720  1.6528
[1] 0.4821507  # sample SD of differences

stripchart(d, pch="|")
abline(v=0, col="green")


Furthermore, even if your expenditure data are not normal, you could use a (nonparametric) paired Wilcoxon (signed-rank) test to see if the differences are centered at $$0$$.

wilcox.test(d)

Wilcoxon signed rank test
with continuity correction

data:  d
V = 39944, p-value = 1.098e-06
alternative hypothesis:
true location is not equal to 0


Ignoring pairs to do (inappropriate) two-sample test: nowhere near a significant result.

t.test(x1, x2)

Welch Two Sample t-test

data:  x1 and x2
t = -0.87948, df = 697.33, p-value = 0.3794
alternative hypothesis:
true difference in means is not equal to 0
95 percent confidence interval:
-0.4347750  0.1657654
sample estimates:
mean of x mean of y
4.532307  4.666812


From parallel stripcharts we can see that purchases were more variable in the second month, but without pairing, any difference in means gets lost in the variability of the samples.

summary(x1);  sd(x1)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
-0.9581  3.0760  4.6222  4.5323  5.9310 10.2613
[1] 1.991526
summary(x2);  sd(x2)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
-1.071   3.187   4.810   4.667   5.910  11.305
[1] 2.054292

stripchart(list(x1,x2), ylim=c(.4,2.6), pch="|")


stripchart(list(x1,x2), ylim=c(.4,2.6), pch="|")