Sample size. Paired T-Test and repeatability

Question

The study involves 10 subjects, in which we measure speed with 4 different shoes, and the measurements are repeated 4 times, and this is done on two different tracks, indoor and outdoor.

On the one hand, we are only interested in reporting if there are differences for all measurements, between indoor and outdoor. That's why I understand that it is a paired T-Test. It would be correct?

In that case, how could you calculate the sample size? I understand that the sample size in this case would be equal to 160, 10 x 4 x 4.

Furthermore, how should I calculate the sample size for repeatability? That is, if I have 10 subjects, as they have repeated with 4 shoes, in reality the sample size would be 40, and the repeated measurements would be 4, right? How many times would it really have to be repeated for that sample of 40?

I've seen some bland altman papers for within-subject standard deviation.

Answer 1

No, it's not a paired t-test. For one thing, you have four different shoes, not two.

I think these data should be analyzed with a multilevel model where the dependent variable is speed, the independent variables are shoe and type of track (and maybe their interaction) and the repeating variable is person.

There is no single definition of sample size here. You have 40 people but 160 observations.

I don't know what "sample size for repeatability" means.

Answer 2

The only way to shoehorn your example into the standard paired t-test framework is to average the indoor and outdoor speeds across shoes and trials for each runner. This gets you down to N=10 but does satisfy all the test assumptions. The averaging may smooth the times enough that this is not a big hit in terms of power.

Another option is to calculate the mean indoor-outdoor difference but then adjust the standard errors for the ten runner clusters. This will give you a confidence interval for the difference. If it contains zero, you cannot reject that there is a track difference. If it does not, then you can reject the null.

Regression of speed on controls for shoe, runner, track, and (maybe) trial, while clustering the SEs by runner, may be a middle ground to doing an a full-blown LMM. The null hypothesis here is that the coefficient on track type is zero.

Answer 3

A few thoughts. First I agree with Peter's assertion that this is not a paired t-test. Personally, I would conduct a mixed-effects regression analysis. For example,the following model might be used for your research with $y_{ijkt}$ being the average of each participant's four speed measurements. If you wanted to test for change over time you would need to add a variable to capture which of the four measurements it was for each combination of shoe and track variables). Regardless, your sample size would be 10 as the sample size = the number of subjects participating. This is a relatively small sample size and could adversely impact both power and generalizability of fixed effects and the ability to precisely estimate variability between individuals. Ideally, I would try to get more participants regardless, but if you cannot, then I would suggest including the measurement variable ($Run_t$) to mitigate the concerns created by a small sample size.

Therefore, there would be 10 participants, four shoes, and two tracks, meaning that each of the 10 participants will have 8 different combinations of shoe-track conditions. With 4 runs per condition combination this results in a total of 320 observations, however one should not confuse sample size and number of observations. Also, you could reduce the complexity of this model at the expense of power and precision by averaging the 4 speed measurements for each shoe-track combination. Though if you care about fatigue or learning-effects, do not average the 4 measurements.

$\beta_n$ = fixed effects (effects of run number, shoes and track that apply to all runners), $u_i$ = random effects $i= runner$

• $y_{ijkt}:$ Speed measurement for subject $i$, shoe $j$, track $k$, and repetition $t$.
• ${Shoe}_{j}$: Dummy variables for the 4 different shoes. (Note, this is represented as three separate binary variables. E.g. for $Shoe_1j$ j=0 or 1 with 0 being the reference level (Shoe A), and 1 being Shoe B, for $Shoe_2j$ j=0 or 2 representing Shoe A and Shoe C respectively, for $Shoe_3j$ j=0 and 3 reresenting Shoe A and Shoe D respectively.
• $Track_{k}$: Dummy variable for the 2 different tracks (indoor and outdoor).
• $u_i$: Random intercept for subject $i$.
• $u_{ij}^{Shoe}$: Random slope for the interaction between subject $i$ and shoe $j$.
• $u_{ik}^{Track}$: Random slope for the interaction between subject $i$ and track $k$.
• $\epsilon_{ijkt}$: Residual error term.

Model without interaction terms $ y_{ijkt} = \beta_0 + \beta_1 \text{Shoe}_{1j} + \beta_2 {Shoe}_{2j} + \beta_3 {Shoe}_{3j} + \beta_4 \text{Track}_{k} + \beta_5 \text{run}_{t}+ u_i + \epsilon_{ijkt} $

However, it should be noted that there will likely be some interaction effects between the individual runner variable, the shoe variable, and the track variable.

$y_{ijkt} = \beta_0 + \beta_1 \text{Shoe}_{1j} + \beta_2 \text{Shoe}_{2j} + \beta_3 \text{Shoe}_{3j} + \beta_4 \text{Track}_{k} + \beta_5 (\text{Shoe}_{1j} \times \text{Track}_{k}) + \beta_6 (\text{Shoe}_{2j} \times \text{Track}_{k}) + \beta_7 (\text{Shoe}_{3j} \times \text{Track}_{k}) + \beta_8 \text{Run}_{t} + [u_i + (u_{i1}^{\text{Shoe}_{1}} \cdot \text{Shoe}_{1j}) + (u_{i2}^{\text{Shoe}_{2}} \cdot \text{Shoe}_{2j}) + (u_{i3}^{\text{Shoe}_{3}} \cdot \text{Shoe}_{3j}) + (u_{ik}^{\text{Track}} \cdot \text{Track}_{k}) + (u_{it}^{\text{Run}} \cdot \text{Run}_{t}) + (u_{ijk}^{\text{Shoe} \times \text{Track}} \cdot (\text{Shoe}_{j} \times \text{Track}_{k}))] + \epsilon_{ijkt} $

As for how to actually conduct mixed-effects regression, I will leave that to you to search as the process depends largely on what software package you're using. However, here is a link to a guide for doing it in R: https://www.learn-mlms.com/index.html

Answer 4

As previous responses pointed out, the fundamental reason why you can not use a paired test is that your samples are not independent (the data from the 4 different shoes was done by the the same subjects). If different subjects had used the different shoes in the 2 tracks, then you could (and should) have used a paired t-test.
Now, what you could do (but not the best choice) is run 4 paired t-test (1 for each shoe type), comparing the times on both tracks. Of course you should use an appropriate multiple comparison correction (Sidak?) to deal with this situation. But you run into the risk that some results will be significant for some shoe(s), but not for other(s). It is not clear then what to conclude. But I would still do it, if only just just for fun, or rather to get a feel for the data, and what it might say. At a minimum, I would compute the 4 paired differences, and plot that data, with the confidence intervals of the 4 means, again just to visualize the data. That alone will tell you a lot (e.g. whether there are clear differences, or not, whether the track, or shoes, make a difference, etc.).
As also already pointed out, you could run a LMM. But you seem uncomfortable with this. A final approach (and the one I would adopt), is to run a factorial ANOVA (aka known as a factorial DOE). You have 2 factors (the 4 shoes, the 2 tracks), and 4 replicates for each combination of factors, with one response (the time taken to run the course). Note that this factorial ANOVA will generate a LMM model, tell you which factor(s) are significant, give you factorial plots, will tell if there are interactions between your facftors, etc. ANOVA under the hood is a LMM, so you will get the same results, but you may be more familiar/comfortable with that approach. This is what I would use, because the software I use will set up the LMM more easily and give more results (plots, response optimizer, etc.).
To answer your specific question about sample size, your total sample size is $10*4*2*4=320$, but you only have $4$ samples (replicates) per combination (a given subject, with a given shoe on a given track).