I have 2 samples, named "1" and "3":
- (1) n = 1282, mean = 14.77, sd = 8.27
- (3) n = 5360, mean = 15.88, sd = 8.55
Their distribution look:
ggplot(creu, aes(x=creu$V7, fill=creu$V1)) +
+ geom_histogram(binwidth=3) + theme(legend.position="none")
I want to show the histogram in a research paper to suggest a possible tendency in group 1 to have lower values than group 3.
Data isn't completely independent so my plan was to show the histogram with a t.test:
t.test(V7 ~ V1, data=creu)
Welch Two Sample t-test
data: V7 by V1
t = -4.3086, df = 1986.994, p-value = 1.723e-05
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-1.6240990 -0.6080744
sample estimates:
mean in group 1 mean in group 3
14.77288 15.88897
I'm aware that significance is highly influenced by the sample size, but considering that this is not the core of the research, just an hypothesis, I don't want to put too much weight on it, simply complement the plot with some statistical data.
- given this conditions (unequal sample size, but very large dataset, not independent and their representation). Is this test the best option?
- Should I add additional information such as effect size (e.g. cohen's d)?
EDIT: Why I said they aren't independent? I took 82 models based on 82 datasets and test each model on the remaining 81 datasets (all but the one used to train it). I expect to see a bad performance in those tests, but for ~19% of them (n=1282) I see a good performance. I compute a "similarity distance" between each pair (dataset-of-the-model & dataset-of-the-test) to hypothesize that maybe those that obtain a good performance are closer, and that's why they can predict reasonable good. I say they aren't independent because maybe model X based on dataset X performed well on dataset A and D but bad on dataset B, C and E. So somehow model X is "behind" both groups.