1
$\begingroup$

There are two groups of samples. The number of events on n-th day is measured for each sample. The average is measured for each day for each group of samples

Days after first event average events count group 1 average events count group 2
1 1.0000 1.0001
2 0.1000 0.1050
3 0.0500 0.0600
4 0.0050 0.0610
5 0.0005 0.0115
6 0.0006 0.0080
7 0.0001 0.0020
8 0.0003 0.0060
9 0.0001 0.0030
10 0.0000 0.0020
11 0.0000 0.0010
12 0.0000 0.0015

The averages start at about the same value, then drop rapidly for group 1 and drop slower for group two. There is some noise "day 6 for group 1" and "day 4 for group 2". Groups are clearly statistically different: on any day the value in group 1 is lower that in group 2 for the same day. However, the t-test shows very small significance (p>0.94) because the value for the first day is very large and almost equal.

What is the correct metrics to show that "two is always greater than one on any given day"?


Update


I've recorded the averages and the stdevs for each day for samples in each group. Here is what I get

enter image description here

Note that at day one every sample is guaranteed to have an event at least once. After that only a small fraction of samples have an event. On each day 0 < avg2-avg1 < (stdev1 + stdev2). With (stdev1 + stdev2) ~ 30 * (avg2-avg1) and more than 30 days of observations.

Should I use this data, and if so - how? Should I just count the number of samples with events on n-th day?

$\endgroup$
4
  • $\begingroup$ Any statistically valid way begins by recording the counts of events along with the sample sizes, because the averages do not reliably reveal the sampling variance of the data. $\endgroup$
    – whuber
    Jul 18, 2022 at 14:39
  • $\begingroup$ @whuber I've updated the question with standard deviations. $\endgroup$
    – sixtytrees
    Jul 18, 2022 at 15:12
  • $\begingroup$ That helps. But the standard errors are the relevant statistics and having the actual counts would be even better. $\endgroup$
    – whuber
    Jul 18, 2022 at 15:27
  • 1
    $\begingroup$ @whuber, the actual counts are known: 640 and 130 for groups 1 and 2 respectively. The number of samples in the group stayed the same during the whole experiment. $\endgroup$
    – sixtytrees
    Jul 18, 2022 at 15:34

1 Answer 1

0
$\begingroup$

Should use nonparametric approach: Mann–Whitney U test or Wilcoxon signed-rank test. It does not make assumptions about the shape of the distribution.

$\endgroup$
2
  • $\begingroup$ What about the dependence between the observations in these time-series data? $\endgroup$
    – Dave
    Aug 11, 2022 at 13:23
  • $\begingroup$ @Dave This is a pairwise comparison between two cohorts on a given day. For each day. It is important that on any given day the second cohort has higher value than the first cohort in a statistically significant manner. $\endgroup$
    – sixtytrees
    Aug 19, 2022 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.