1
$\begingroup$

I have data sets that represent multiple measurements over time. The data sets come from biological experiments in which we measure something in N individual cells, with around 250 measurements over a time frame of around one minute. For each of the time points, we have summary statistics (mean, SD, N) that we can plot to create a curve; this curve will thus be the average curve of, say, 30 individual curves from individual cells, and error bars will be either the SD or SEM of those individual curves.

Now, we have several such "average" curves that represent, e.g. "treated" versus "non-treated" cells. We need to statistically evaluate whether the curves are significantly different.

What I have done so far are multiple t-tests ("one per row") between two curves - this gives me individual p values, one for each time point, which looks like a reasonable start. In principle it tells me from what time point in the measurement the curves are significantly different (if they are at all).

Is there a tried-and-true statistical method to get a single p-value to determine whether two such curves are significantly different? Can anyone let me know (in layman's terms) how to do it?

edit: For illustration, here is a typical result - we have means, SD and N for every time point, but show only a few SDs here as error bars to not totally obscure the "average" line. What we'd like to have is one value that describes the significance of the difference between two lines, e.g. of the blue and the red ones.

three example graphs

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
0
$\begingroup$

Is there a tried-and-true statistical method to get a single p-value to determine whether two such curves are significantly different?

Unless you have some theoretical form for the change in signal over time, this seems to be an ideal application of generalized least squares regression. You model the individual observations as a flexible function of the combination of time and treatment, doing so in a way that takes the correlations of observations within individual cells into account. That is sometimes called the "growth curve model," a name that seems to describe your example data pretty well. Frank Harrell's course notes and book explain the approach with an extended example in Chapter 7.

Using 250 separate t-tests is both inefficient and potentially misleading. With a p < 0.05 cutoff for "significance" you would expect about a dozen false positives just by chance. Also, that doesn't directly model the time course or account for the fact that you have repeated measurements on individual cells.

You can accomplish what you want with a single generalized least squares model. Each observation is modeled, with one row per observation notated with time, treatment, a cell ID, and any other covariates you want to evaluate.

The time course is fit with just a few coefficients, for example via a regression spline, versus the 250 separate coefficients you implicitly use now. With fairly simple, non-wiggly curves as you show, you only need a few "knots" to break the time axis into time periods for this. Within each time period between a pair of knots, values are fit with a cubic polynomial. At each knot, the polynomials from the time periods on both sides are constrained to be continuous and smoothly joined. From the data you show you might even be able to fit a single low-order polynomial over the entire time range. The idea is the same whether you use a single polynomial or a spline: describe the time course with just a few coefficients whose values you estimate directly from the data.

To evaluate the effect of treatment, you include as predictors in the model both a term for treatment by itself and an interaction between treatment and time, including interactions with the spline/polynomial terms modeling time. The coefficient for treatment would determine whether there was any difference between treated and control at a reference time point, typically at time = 0. The interaction terms allow the shape and time course to differ between the treatments.

To account for the correlations of observations within individual cells over time, you specify an overall structure for those correlations that is incorporated into the modeling. The Harrell references show how to do that in the context of generalized least squares. Mixed models with "random effects" for cells might be used instead to this end, but I don't see any advantage for your application.

"To get a single p-value to determine whether two such curves are significantly different," you do a test that evaluates all of the coefficients associated with treatment to determines whether any are different from 0. If you work with the tools in Harrell's rms package and its Gls() interface to generalized least squares, that's readily done with that package's anova() function. If you have a particular interest in the differences at 50 seconds, you can get error estimates that include information from all of the observations, not just those taken at 50 seconds.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you very much for the detailed answer - there is quite a bit I first have to digest before I can get into action. One thing I do not (yet) understand is how, from modeling the curves (that is easy in my case) I can get to a statistical evaluation. In particular, I have the feeling (correct me if I am totally wrong) that by modeling the curves I will lose the information from the repetitive measurements per time point (the error bars/ standard deviation and n). Disregarding this information cannot be correct... $\endgroup$
    – frofa
    May 15, 2022 at 16:11
  • $\begingroup$ @frofa if you use all the individual observations, annotated as I suggest, then the statistical model will incorporate information from all the observations. You don't lose anything, as all of the observations contribute to estimating the coefficients that define the curves. $\endgroup$
    – EdM
    May 15, 2022 at 16:29
  • $\begingroup$ Thanks again. We do, of course, have all individual observations, but as we already have the summary statistics, we would very much like to work with those instead. This would scale much more easily to a larger number of experiments, and also allow to use normalized data (as we do to create the average curves and error bars in my example). $\endgroup$
    – frofa
    May 16, 2022 at 10:54
  • $\begingroup$ @frofa using the summary statistics time point by time point discards all the information about trajectories over time within each cell. The data-storage and computational requirements to work with individual observations don't seem to be overwhelming here. Including the time 0 observation as a covariate helps normalize, and you can display 0-1 "normalized" data after the statistical tests based on generalized least squares. $\endgroup$
    – EdM
    May 16, 2022 at 11:24
  • $\begingroup$ @frofa I suppose you could consider weighted least squares (WLS) while modeling treatment and time as suggested for generalized least squares. That's a "specialization of generalized least squares." You weight each summarized time-point value by its estimated variance. But, depending on the nature of the data, that's not only an inefficient use of the data but it can also be misleading, as it ignores the correlation structure within the data. $\endgroup$
    – EdM
    May 16, 2022 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.