1
$\begingroup$

I am unsure how to interpret the significance of a paired test.

I have data for six days of the week; for each day I have relevant data for 11 hours minutewise. I want to determine, whether a trait 'A' is more common on different days.

Both, the total number of observations N as well as the number of observations of trait 'A' vary with the time of day in a very similar fashion on the different days.

I proceed as follows: For every minute, I compute A/N. I then perform a series of paired t-Tests (or Wilcoxon-signed-rank tests) to compare the ratios A/N of different days with the time as pairing factor.

I intepret a p-value below a given threshold (taking into account a Bonferroni correction) as A/N being significantly different on the days I compared.

What puzzles me is that if I look at the ratios A/N aggregated over the entire day, I find Tue > Fri > Sat. But while the differences between Tue and Fri as well as Fri and Sat are significant, the difference between Tue and Sat is not significant.

Is the interpretation that a significant paired test signals a difference in means correct?

Is it too naive to expect the significance of paired tests to be transitive when looking at aggregated values?

Edit: I have a nominal variable and trait 'A' is one level. I want to compare the frequency, with which 'A' occurs in the total number of observations. The sampling times are the same for all days and are known precisely, I binned them into minutes. I have one data point per minute per day.

Here is a plot of the data, all figures are per minute: I want to test, whether there are difference in the relative number of occurences on the different days. Since the total number of occurences also varies on the different days, I form the ratio A/N and analyse that.

$\endgroup$
  • 1
    $\begingroup$ Can you give some more information? What is the trait? Sample sizes? How many obs ---per day? per minute? Is the distribution of sampling times over the day equal for the different days? $\endgroup$ – kjetil b halvorsen Mar 10 '17 at 9:52
  • $\begingroup$ @kjetilbhalvorsen I have a nominal variable and trait 'A' is one level. I want to compare the frequency, with which 'A' occurs in the total number of observations. The sampling times are the same for all days and are known precisely, I binned them into minutes. I have one data point per minute per day. $\endgroup$ – Neuneck Mar 10 '17 at 10:02
  • 1
    $\begingroup$ Then, very probably there is some autocorrelation in the data. Can you comment on that? What is the correlation time? Can you augment your post with a plot of the autocorrelation function? $\endgroup$ – kjetil b halvorsen Mar 10 '17 at 10:05
  • 1
    $\begingroup$ OK. The number of observations in the figure (Total " obs / 100), is that pr. minute or per hour or what? From the graphs, the event prob is larger at start/end of day, so series is not stationary. Even if you doubt autocorrelations, thats OK, statisticians would want to check for it, but since series is not stationary, that would need some simpel prelim model (maybe simply smoothing) and autocorrelation of residuals. $\endgroup$ – kjetil b halvorsen Mar 10 '17 at 10:36
  • 2
    $\begingroup$ Thank you. Can you add *all this new information to the original post? So the rate is quite high then. I would (try to) treat this with (some kind of) logistic regression, then (with overdispersion), and I would certainly investigate possible autocorrelations. $\endgroup$ – kjetil b halvorsen Mar 10 '17 at 12:17
-1
$\begingroup$

A paired test checks, if the distribution across the uniting variable is the same or not. Hence, a paired test can be significant for two variables with the same overall mean.

Therefore, the interpretation that a significant paired test implies a difference in means is wrong. The converse is true however, if two variables are drawn from the same distribution, their means are bound to be the same.

To test, whether the sample means are different for non-normal distributions such as the one here, a non-parameteric test like the Mann-Whittney-U Test is appropriate and produces consistent results.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.