I have data where every record has an ID, a numeric value (that represents the number of times each ID took a particular action) and a binary state that represents whether they achieved a goal.
id action goal
1 2 FALSE
2 41 TRUE
3 37 TRUE
4 1 FALSE
5 5 FALSE
6 6 FALSE
7 64 FALSE
8 4 TRUE
9 13 FALSE
10 6 FALSE
... ... ...
It's approximately 10,000 records. I want to compare if there is a statistically significant difference for the mean number of actions in the group where the goal is true versus the group where the goal is false.
My null hypothesis is there is no difference between the two groups. I'm interested in checking both tails, whether one group is larger or smaller than the other group.
I tried the analysis of variance to test the null hypothesis.
summary(aov(as.numeric(goal) ~ action))
Df Sum Sq Mean Sq F value Pr(>F)
action 1 2 3.9853 16.06 0.000985 ***
Residuals 10000 2100 0.1795
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
My understanding is that this says there is statistically significant evidence to reject the null hypothesis.
I have a few questions. a) is this the right test for a two-tailed test to check the difference of means from the same sample? b) am I reading it correctly? c) what does it mean that they are significantly different? does it mean they come from different populations?
The reason that I ask b is that the distribution looks nearly identical. The mean number of actions when the goal is TRUE
is 46.85 and when FALSE
is 37.95.
Additionally, the distributions look like they cover right on top of each other.