Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In a longitudinal study, two groups of subjects have been measured over a period of two years at 6 months intervals. During these measurements subjects have been assessed with a series of $k$ measures ($M_1$ to $M_k$).

From other sources we expect to see a difference between the groups and a decline in the performance of only one group. The question is: which of the $M_1$ to $M_k$ measures can pick up the decline of performance over the time.

If we had only one measure M, I suppose we could simply get an answer using:

fit.0 <- lme(M ~ Group * Time, random=~1|Subject)

Then we could simply check the significance of the interaction effect. However, I guess looking at $M_1$ to $M_k$ measures one by one might not be ideal. Although differences between the two groups are assumed to be real according other sources, still some form of multiple comparison might be going on.

Another way to look at the problem might be to try to address our research problem directly, i.e. trying something like this:

fit.1 <- lme(Group:Time ~ Group + Time + M0 + M1 + ... Mk, random~1|Subject)

This way, I could directly see which $M_1$ to $M_k$ terms is a significant predictor of the interaction term (or I could add the $M_1$ to $M_k$ terms one by one or use some other model selection method). However the main problem is that it does not work! It seems R does not like to have an interaction term as the dependent variable.

So here is what I want to know: is the basic idea behind the second approach sound? If yes, is there a way to make fit.1 work?

Comment 1: The problem of finding the measures that can predict decline in longitudinal studies is not that rare, at least not in my corner of the world. For instance consider the case of neuro-degenrative diseases like Parkinson's or MS. From all kind of sources we know that these diseaes progress constantly and slowly. Thus it is interesting to find the measures that reflect the progress of the disease.

share|improve this question
    
How much data have you got? Estimating measure-dependent main effects and interactions may be possible (using e.g. Seemingly Unrelated Regression or some GEE analog) but the wisdom of trying this approach depends on k, and your sample size. –  guest Feb 24 '12 at 22:35
    
Yes, that's a good point. As usual for the clinical studies the sample size is relatively small. I understand the problems with estimating too many parameters in small dataset. Thanks for pointing it out. –  AlefSin Feb 25 '12 at 0:47
add comment

2 Answers 2

up vote 1 down vote accepted

I don't think your second approach makes sense as it is put. Group*Time is really a statement of your experimental design and I can't conceptualise a statistical model whereby that is the random variable explained by a bunch of measures that come out of the experiment.

However, I can see a possible sort of way out which is related to your thinking there. You have two issues here. 1) Is there really a decline in performance in one of the groups; and 2) which is the best measure. Most commonly of course in this sort of situation we are worried about just question 1; but your research question seems to be 2.

The problem comes about from trying to answer both questions simultaneously. I think that if you could put aside question 1, you would have a better chance of answering 2. That is, how confident are you from prior knowledge that there really is a decline in performance? Is there any outside evidence which suggests how strong that decline is? And are you confident that in the other group it really is no change?

If you have this degree of confidence you could construct a new response variable which is what you expect to see in each groups performance ie for one group each of the four values is the same; for the other it declines at X% per six months. Then you could make this the response variable in something like:

theoreticalchange ~ m1, random=~1|Subject+Time

Where the Time ra ndom effect is meant to pick up the uncertainty in your guess as to what the true level of change is. There's no group random effect because that would be completely correlated with the theoretical change.

I don't think you want to put all six measures in the same model, as implied in your "approach 2". They are probably highly correlated, and you will be none the wiser about which is the best predictor in its stand-alone state (which sounds like your research question). You could fit each of your six candidate models, and choose the one with the best BIC or AIC. I can't think of a rigorous statistical test though of which of your six measures is better - the problem is there is no way of nesting your candidate models in a hierarchy, so you have all the usual problems of model selection with a bunch of non-nested candidate explanatory variables.

For this to work, you would still need to have enough data; you would need a lot of confidence in your theoretical construct (ie you are focused purely on Q2, and Q1 is answered for you); and you need to choose a method of model selection. So a bit of a long bow - but you will get an answer and at least a pragmatic sense of how good each of your indicators is at predicting the expected theoretical change.

I'm pretty tentative about all this however; I haven't seen anything similar to this (making a theoretical, unobserved construct the response variable in a statistical model) in a reputable analysis, so I'll be interested to observe any comments that come.

share|improve this answer
    
thanks a lot for your extensive answer. Sorry for getting back to you with so much delay: I have been traveling these days and didn't get a chance to comment here. –  AlefSin Feb 29 '12 at 20:05
    
As you have pointed out, I'm mostly interested in Q2 as Q1 has been addressed using a different set of measures elsewhere and also all evidence and literature points to that. Unfortunately I don't have a huge sample so I guess regarding Q2, I'm going to focus mostly on identifying those measures that can show the decline rather than ranking them. –  AlefSin Feb 29 '12 at 20:13
    
I liked the approach you have pointed out. I wonder how you compare it with the fit.0 in my question: I saw an instance that the Group*Time interaction was not significant with fit.0 but was very significant with your method. I wonder how to explain it. Also I assume in your model by random=~1|Subject+Time you actually mean random=~1|Subject/Time as the way you have written it gives a syntax error. –  AlefSin Feb 29 '12 at 20:18
add comment

Another option is structural equation modelling using a latent growth curve model. Your latent variables would be slope and intercept which would be predicted off your series of M measures. You would then check the slopes, intercepts, and means over time.

share|improve this answer
    
thanks a lot for your comment. I'm not very fluent in SEM though I can guess how a model like what you suggested might work. I'll have to spend some time on it to see how I can build and evaluate such model. –  AlefSin Feb 29 '12 at 20:20
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.