Multiple measurements of a single datapoint: what to do with variability?

Question

I am interested in a general question about research design, but I will explain it on a specific example.

Suppose I want to determine if one group of people has higher heart rate than the other group. Let's consider two methods to do that, two scenarios:

Scenario 1: we get one measurement of heart rate from 5 people in each of the two groups. Then for each group we can calculate the mean and standard deviation, and run a t-test with that data to see if groups are different.

Scenario 2: we get several, let's say 3, measurements from each of 5 people in each of the two groups, then average the 3 measurements for each person and use it a single datapoint for this individual. Then proceed as in scenario 1.

I want to understand if the second scenario is valid. Such a design is commonly used and it seems to reduce the variability compared to the first scenario. But I am very concerned that in scenario 2, we are simply discarding, or ignoring, the intra-individual variability.

So is it ok to do that? What is the cost of this? Can this lead to incorrect conclusions (false-positive, false-negative)? Or maybe the conclusion has to formulated in a very specific way? What exactly are we characterizing with this method? Average heart rate of a person, as opposed to a random heart rate reading of a person? Is it what we should be interested from a biological standpoint?

Sorry, if the question is not totally clear, but it is not clear to me either, and may be you can point me in the right direction. Any thoughts are welcome.

Update: I just realized that there is a third scenario: start as second, but use each measurement (not each person) as a datapoint, which increases the number of observations 3-fold. What are the advantages or limitations of doing that?

Answer 1

Your concerns are legitimate.

Scenario 1 is valid, but unless you have a very large number of people in your group, your results won't reflect "real" difference or they might be biased. For example, the first time you take heart rate, the subject are stressed by the procedure and all have a very high heart rate. By taking more than one measurement, you reduce that bias.

Scenario 2 is better, altought by making a mean you lose information. You might be more precise in your evaluation of heart rate, but you won't take into consideration all the heart rates possible in the 2 groups.

I usually use scenario 3 but the way you describe the analysis is pseudoreplication. I recommend the classic ecological article of Hulbert, to read on the subject : Hurlbert, S. H. 1984. Pseudoreplication and the design of ecological field experiments. — Ecol. Monogr. 54: 187-211.

Now, there is a way to use all the information and not being pseudoreplicated. You can use a mixed model (so basically, an ANOVA), with in the fixed terms the group and in the random term, the ID of the subject. I hope I used the correct terms, as sometimes I get mixed up. If someone see a mistake, please correct me.