2
$\begingroup$

I want to analyze data from same patients in two timepoints. The problem is the measurements have high variability between patients and it makes more sense to first adjust the observed value to the baseline, and compare them afterwards.

This way however I am losing the baseline timepoint, and I am a bit unsure if a t test would be ok to use in such situation.

[Question1] Is it correct to use a t test when comparing with baseline values of 1?

[Question2] Is this basically a one sample t test?

Here is a reproducible example using iris dataset. I modified the data to represent my problem.

pacman::p_load(tidyverse,ggpubr)

df <- iris %>% filter(Species %in% c("virginica", "versicolor")) %>%  # pretend that these two species are coming from same subjects
mutate(patient_id = rep(1:50,2)) %>% # add the subject ids to the observations
 mutate(timepoint=c(rep("pre",50),rep("post",50))) %>%  # pretend that the measurements were done in two timepoints and are from the same subjects
 filter(patient_id %in% c(1:36)) %>% # take only the first 36 subjects
 dplyr::select(patient_id,timepoint,Sepal.Width) %>% # keep only the relevant columns
 group_nest(patient_id) %>% # nest the data by subject
 mutate(relative_change = map(data, function(x) { # calculate the relative change from baseline measured value for each subject
  x %>%
   mutate(relative_change = Sepal.Width/Sepal.Width[timepoint=="pre"])
 })) %>%
 unnest(relative_change) %>%
 dplyr::select(-data)

# plot the data without taking the baseline values in to account
df %>%
ggboxplot(x = "timepoint",
          y = "Sepal.Width",
          add = "jitter")+
 stat_compare_means(comparisons = list(c("pre","post")),
                    method = "t.test",paired = T)

# plot the data with values relative to the baseline values
df  %>%
 ggboxplot(x = "timepoint",
           y = "relative_change",
           add = "jitter")+
 stat_compare_means(comparisons = list(c("pre","post")),
                    method = "t.test",paired = T)

# Is this formally ok to do?
t.test(df$relative_change[df$timepoint=="pre"],df$relative_change[df$timepoint=="post"],paired = T)

# This basically is a One sample T test right?
t.test(df$relative_change[df$timepoint=="post"],mu=1)

enter image description here

Edit to include more background information on the experiment as requested in the comments:

The data comes from two experiments where each individual's cells were exposed to certain conditions. The experiments are quite labile and depend strongly on the cells and the background noise which are both quite stable per patient. Therefore, the results of measurment "pre" to "post" in each patient is much more reliable than if we would compare multiple "pre" to multiple "post" reads, that come from noisy measurments.

We are interested in the change of post to pre.

Nevertheless the noise is introducing a large variation in the reads. Therefore my idea was to:

  1. first calculate the ratio of post/pre and then
  2. check statistical inference.
$\endgroup$
5
  • 3
    $\begingroup$ There is no other relevant information about the patients in addition to the two measurements at two different time points? Then use matched-pairs t-test on the original measurements. $\endgroup$
    – dipetkov
    Oct 22, 2023 at 9:34
  • $\begingroup$ @dipetkov well there is: the error of the measurement itself which is smaller when pre to post values per each individual are compared. $\endgroup$
    – WojciechF
    Oct 22, 2023 at 15:40
  • $\begingroup$ This simply says that the within-patient variability is smaller than the between-patient variability. This tends to happen because taking the pre-post difference by patient cancels the effect of any patient characteristics that don't change between the two time points but otherwise are predictive of the condition (say previous history of illness). It is the reason to use a paired test when we have paired observations. What I meant is if you have data on relevant covariates and a treatment, in short why are you doing this pre-post comparison. $\endgroup$
    – dipetkov
    Oct 22, 2023 at 17:04
  • $\begingroup$ stats.stackexchange.com/questions/3466/… $\endgroup$ Oct 22, 2023 at 17:37
  • $\begingroup$ Thanks for the update! The extra information is helpful though I think there remain a couple of points to clarify: What are the two experiments & conditions? Are you interested in comparing differences across experiments / conditions? It seems your data is reads per cell; are there multiple reads per patient per timepoint in each condition? Do you need to apply any standardization steps to the cell reads? The more information you provide, the more relevant the answers will be. If you can replace the iris data with data that is representative of your cell experiments, that's even better. $\endgroup$
    – dipetkov
    Oct 23, 2023 at 11:50

4 Answers 4

4
$\begingroup$

When plotting your data, you want to show as much detail as possible, including the individual patients. So create a before-after graph, where each patient is shown as two connected points.

A paired t-test tests the null hypothesis that the difference between before and after measurements, on average, is zero. It is the same as a one-sample t-test on the set of differences, with a null hypothesis of zero difference.

Your second graph plots *relative change", which seems to be the ratio (not the differences) of the two values. To answer your question: No, a paired t-test on the raw data is not the same as a one-sample test on the set of ratios. It is the same as a one-sample test on the set of differences.

If you care about the ratios, one way to do analyze the data is to take the logarithm of all the values, and then run the paired t-test on the set of logarithms. The difference between log(values) is the same as the log(ratio).

A final note. Before getting immersed in statistical analyses, look at the big picture. Your graph shows that a huge fraction of the patients have an increase and a huge fraction have a decrease. You may be able to show statistically that the excess of increases is unlikely to be by chance. But the big picture is many go up, many go down. Not sure what conclusions you'll be able to draw.

$\endgroup$
2
$\begingroup$

It's likely the wrong test to use to assign causality to the exposure. You should not be comparing pre- with post-exposure, you should be comparing how things are post-exposure (or the change from pre- to post-) when exposed to the exposure of interest compared to e.g. nothing (i.e. a control condition/exposure that should do nothing). Only if you knew with absolute and complete certainty that the change from pre- to post-exposure would be exactly 0 would a paired t-test (or a t-test of the change - the two things are almost the same thing) be a sensible choice. If you don't know with complete certainty, then a randomized controlled experiment with a control group would be one traditional option (could be analyzed e.g. using ANCOVA aka a linear model with model terms for exposure as a factor and pre-exposure levels as a covariate). In all cases, adjusting for the pre-exposure level can be a good idea (it frequently is).

$\endgroup$
1
$\begingroup$

If you have no other information about these subjects, then a paired t-test is fine. This is equivalent to a one sample t-test on the difference between pre- and post-.

set.seed(1234) #Set a seed
pre <- rnorm(100)
post <- pre + rnorm(100,0,10)
t.test(post, pre, paired = TRUE) #t = 0.39957, df = 99, p-value = 0.6903
change <- post - pre
t.test(change) # t = 0.39957, df = 99, p-value = 0.6903

I don't know what you did to get the 2nd graph (I don't use tidyverse) but it looks like you had to torture the data a lot, so I am not going to use that. Much better if you could tell us about your actual data.

If you have more information about your cases (apparently they are patients, since you use that term) then there may be better options, but what they are depends on the particulars of your data and your research question.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you for taking the time to answer. The only torture I did to the data in the right plot was this : mutate(relative_change = Sepal.Width/Sepal.Width[timepoint=="pre"]), meanning that I divided the actual value by the "baseline" or "pre" values. Therefore we get 1 on the "pre" and a ratio on the "post" $\endgroup$
    – WojciechF
    Oct 22, 2023 at 14:58
1
$\begingroup$

I personally do not see the point in trying to "erase" variation or minimize it before analyzing your data. The dispersion of your data is a feature of the data in itself.

If you want to compare between "pre" and "post", but adjusting to the baseline means removing "pre", what exactly would you analyze ? T-tests and ANOVAs are analyses of variance. There is no variance in the newly made variable which consists only in 1s. This variable is not even a measurement anymore. So no, it is not OK to do.

In my opinion, the best way to go about your data is to fit a GLM with a distribution from the Poisson family, with the individual as a random factor, since you have 2 measurements per individual. If in your data the variances between pre and post are unequal, and/or there is much more dispersion than in the example you are showing, I would use a negative binomial distribution (glmer.nb function of the MASS package in R).

Edit : Is there a control treatment in your dataset ? Because without such a treatment, you cannot know whether the difference in the values of "pre" and "post" is due to the treatment or simply due to the passing of time.

$\endgroup$
3
  • 1
    $\begingroup$ Thanks for your answer. The point to decrease or "erase" variability (as you called it) in the data is to reduce the noise and find underlying biologically relevant differences. $\endgroup$
    – WojciechF
    Oct 22, 2023 at 15:16
  • $\begingroup$ Thanks for the tip with GLM, I should have thought about it. I am also asking myself if a mixed effects model would fit here to with a random effect of patient_id $\endgroup$
    – WojciechF
    Oct 22, 2023 at 15:23
  • $\begingroup$ My argument is that the noise is data, it is biologically relevant. $\endgroup$
    – CaroZ
    Oct 22, 2023 at 16:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.