Quite a while ago, I asked a question for which Peter Ellis provided a very interesting answer. Now I'd like to follow on that and have your comments and ideas on how to actually put it to use.

I try to restate the problem, hopefully this time in a more clear way using a synthetic example.

Problem: Two groups of subjects participate in a longitudinal study. One is a control group and the another is a group of subjects with a degenerative disease. That is to say, from all other sources we know and expect to see a decline in the conditions of the subjects as time goes by. Over the course of the study, we measure all subject using a series of performance measures $M_{1}...M_{n}$. Our research question is: which of these measures can pickup the decline in patient's conditions the best.

Suggested solution: Peter's insight was to build a theoretical response variable and use model selection methods to see which of $M_{1}...M_{n}$ could predict it the best.

Here is a simple simulation with 2 measures to demonstrate the problem the best:



# Followups in the longitudinal study
t <- 1:5

# Number of subjects in each group and number of followups
N <- 20
M <- length(t)

# Subject IDs
id <- c(paste('C', 1:N, sep=''), paste('P', 1:N, sep=''))

# The first measure. Each subject has a unique value at the baseline.
# There is also a difference between the groups at the baseline.
m1.c.start <- 150 + rnorm(N, 0, 5) 
m1.p.start <- 140 + rnorm(N, 0, 5)

# The control group does not change over the time.
# The patient group, however, decline over the time.
m1.c <- rep(m1.c.start, each=M) + rep(0 * (t-1)/M, N) + rnorm(N*M)
m1.p <- rep(m1.p.start, each=M) + rep(-5 * (t-1)/M, N) + rnorm(N*M)

# Generate the data for the second measure.
# The separation of the two groups and rate of progress are weaker.
m2.c.start <- 70 + rnorm(N, 0, 3) 
m2.p.start <- 71 + rnorm(N, 0, 3)
m2.c <- rep(m2.c.start, each=M) + rep(0 * (t-1)/M, N) + rnorm(N*M)
m2.p <- rep(m2.p.start, each=M) + rep(1 * (t-1)/M, N) + rnorm(N*M)

# Now put everything in a dataframe
data <- data.frame(
  m1 = c(m1.c, m1.p), 
  m2 = c(m2.c, m2.p),
  group=c(rep('C', M*N), rep('P', M*N)), 
  id = rep(id, each=M),
  t=rep(t, N*2))

Here are the plots of measures over the time: enter image description here enter image description here

Now to build a theoretical response: In the case, we'd like to see a constant response over the time for the control group and a change of performance for the patients. Also, at the baseline groups are typically already a bit different.

# The desired, theoretical response
data$theory <- 2 + (data$t-1)/M
data$theory[which(data$group=='C')] <- 1

Here is the plot of the theoretical response: enter image description here

Notice that the exact value of the slope and the differences between the two groups in this theoretical curve should not be important as long as model we fit could consider them.

Finally, we fit a linear mixed model for each measure as well as a null model:

fit.0 <- lme(theory ~ 1, random=~1|id, data=data, method='ML')
fit.1 <- lme(theory ~ m1, random=~1|id, data=data, method='ML')
fit.2 <- lme(theory ~ m2, random=~1|id, data=data, method='ML')

I fit the models with ML option as I'm interested in comparing the fixed effects.

The final step is to compare the models to pickup the best measure of the disease progress. I use $AIC_{c}$ for this purpose. The AICcmodavg has a convenient function to build an $AIC$ table for us:


f <- list(fit.0, fit.1, fit.2)
aictab(f, modnames=c('Null', 'm1', 'm2'))

Model selection based on AICc :

     K   AICc Delta_AICc AICcWt Cum.Wt     LL
m1   4 -45.15       0.00      1      1  26.68
m2   4 119.51     164.66      0      1 -55.65
Null 3 130.22     175.37      0      1 -62.05

The results suggest that $M_{1}$ is the best predictor of the theoretical response here.

Now what I'd like to know is:

  • Do you think the whole idea of this approach to pickup the best measure related to disease progression make sense? Have you seen something similar? Is this the right way to do it?
  • Do you think the theoretical model I built is sound? Are there cases that this model would mislead us?
  • Can we expand the same kind of approach for picking up the best measures in other types of designs (like e.g. before/after interventions, comparing two different interventions)?
  • Any other comments or suggestions?

EDIT: I changed bio-marker to measure in the text as I suspected that saying what we deal with are bio-markers draws attention to issues that are not pertinent to this narrow definition of the problem.

EDIT: With this approach looking at combination of measures is easy. For example:

fit.3 <- lme(theory ~ m1 + m2, random=~1|id, data=data, method='ML')

In this example, the $AIC_{C}$ of fit.3 will be slightly worse than the model with only $M_{1}$ (i.e. fit.1).

  • $\begingroup$ I work in a research lab with scientists that are expert in cancer. Their knowledge is needed to identify the biomarkers. Get this down to a small number. It really then comes down to being a classification problem. To actually solve a real problem like this can take years. Combination of markers may work better and success may depend on the actual genetics for the individual. $\endgroup$ – Michael R. Chernick Oct 11 '12 at 17:19
  • $\begingroup$ I think you and probably Peter Ellis are trying to find a neat statistical solution to a problem that in reality would be more complex and require a series of studies on cell, animals and humans with statistical analysis at each stage. $\endgroup$ – Michael R. Chernick Oct 11 '12 at 17:19
  • $\begingroup$ @MichaelChernick Thanks for your input. I mostly deal with biomechanics where problems like above are encountered all the time. Typically, a new device is built that can extract a large number of features related to subject's performance. The question is which of these measures, putting other validity considerations aside, can separate the groups or show progress of diseases the best. The methods I see in the literature are not satisfactory as they usually use a series of ANOVAs with the measures as DV and then compare effect sizes, etc. $\endgroup$ – AlefSin Oct 11 '12 at 18:37
  • $\begingroup$ @MichaelChernick Often the "expert" knowledge is biased towards what has already been studied (think how many article have been written about the p53 gene). Even experimental studies would suffer from this bias (it's easier to find antibodies to well characterized proteins, etc.) Taking a more data-driven approach might be the ticket in this particular instance! $\endgroup$ – Sameer Oct 12 '12 at 0:05
  • $\begingroup$ Nothing works perfectly. $\endgroup$ – Michael R. Chernick Oct 12 '12 at 0:19

To answer some of your questions:

Do you think the whole idea of this approach to pickup the best measure related to disease progression make sense? Yes. Consider also that it might be a combination of measurements rather than a single one.

Have you seen something similar? Yes. In almost all chronical diseases.

Do you think the theoretical model I built is sound? I think your model is a very ideal one. How good it is depends upon many factors you are not mentioning, like the type of disease, the type of measurements, etc.

Can we expand the same kind of approach for picking up the best measures in other types of designs (like e.g. before/after interventions, comparing two different interventions)? This is a very ambiguous question. In some cases you can while in others cannot.

I believe a good and simple example of a disease progression indicator can be found in HIV/AIDS. Here some papers:

Circulating HIV-1-infected cell burden from seroconversion to AIDS: importance of postseroconversion viral load on disease course.

Natural history of HIV-1 cell-free viremia.

  • $\begingroup$ thanks for your answer. I'm traveling now and have very poor internet access. I'll check the papers you linked later as I can only see the abstracts from here. Anyway, after thinking some more, I believe my suggested theoretical model sucks and is probably wrong as it is not flexible enough to show an arbitrary difference $d > 0$ between the groups and an independent rate of decline $\alpha > 0$ at the same time. I wonder if there any systematic method to build such theoretical models? $\endgroup$ – AlefSin Oct 13 '12 at 10:43

Creating a theoretical response doesn’t seem very promising as it doesn’t exist in practice, on the other hand I would suggest to use a standard statistical model (logistic regression with a random effect on the subject level for example) to see which biomarker predicts the group (treatment versus control) more accurately, this would be the biomarker which is most adapted to your study I suppose.

  • $\begingroup$ But at which time point? Please notice that the objective of the study is to find the best indicators of the disease progression. Finding those that separate the two groups at say, the baseline, do not necessarily help with that. $\endgroup$ – AlefSin Oct 16 '12 at 9:31
  • $\begingroup$ Your gold standard is actually the "group" variable and I think the biomarker which distinguishes the best between patients and controls (simultaneously in all time points) should perhaps be the best among others. $\endgroup$ – Mehdi Gholamrezaee Oct 18 '12 at 9:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.