2
$\begingroup$

I've processed my data in two ways to determine significant difference, one method suggests a very high significant differences (p = 0.0000...), while the other returns non-significant results (p = 0.52). Immediately the second option seems more likely, as having such a small p number is suspicious to me, but if someone could tell me which of the following methods is correct (or if neither are correct please let me know) it would be really appreciated.

Data has been collected from four subjects for two tasks. First task involved monitoring heart rate for 1 min before an exercise task, and another 1 min is monitoring of the heart rate following the task.

Method 1) Take the one minute data set per subject for the 'before' task, and average them, so I have one data set representing the one minute. Do the same for the 'after' task. Data is sampled at 100Hz, so there are 6000 data points per 'before' and 'after' task. Use this data for the T-test and get p = 0.0000 (very small).

Method 2) Take the average of the one minute data set per subject so I have a single value, do the same for all subjects so I have four values each representing the average heart rate of that subject for the 'before' task. Do the same for the 'after' task so I have the average heart rate per subject following the exercise. Take these much smaller data sets (4 per set) and get p = 0.52.

Are either correct, or both wrong? Any suggestions would be great.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Method would depend on what exactly is your research question. Do you want to determine if the heart rate, averaged over 1 minute, increase after exercise task? Or do you want to determine the change in beat-to-beat variation after exercise task? You must have a clear question before choosing the data organization and statistical method. $\endgroup$
    – rnso
    Commented Apr 29, 2015 at 3:59

2 Answers 2

Reset to default
1
$\begingroup$

Your first method gives a p-value that is (probably) too low, as the points are not independent.

Your second method gives a p-value that is (probably) too high, as you have discarded information about the sample.

I would use the first method, but include the fact that the measures are not independent, using a multilevel model or a sandwich estimator. What software do you use?

Improve this answer
$\endgroup$
2
  • $\begingroup$ I have access to SPSS, MATLAB and Excel. Predominantly using MATLAB. $\endgroup$
    – anon
    Commented Apr 28, 2015 at 22:56
  • $\begingroup$ I don't know about matlab, but you can do this with a mixed model in SPSS (or you could use complex samples). You can't do this with excel. $\endgroup$
    – Jeremy Miles
    Commented Apr 29, 2015 at 2:18
0
$\begingroup$

Method 1 seems wrong to me as the data you are comparing are not independent samples, i.e., they come from a time-series obtained from a persons' heart measures and by taking 6000 data points you inflate N sample size. Method 2 seems correct even though it is probably not correct to use a t-test here as the dataset is very small.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you. What method would you suggest to determine significance with such a small dataset? $\endgroup$
    – anon
    Commented Apr 28, 2015 at 20:05
  • $\begingroup$ If it were possible to collect more data that would be best. If you insist I would say use a non-parametric test, e.g., Wilcox test. $\endgroup$
    – Ruthger Righart
    Commented Apr 28, 2015 at 20:12
  • $\begingroup$ Why non-oarametric? $\endgroup$
    – Jeremy Miles
    Commented Apr 28, 2015 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.