I want to study the yearly weight change of a population of about 100 people. I have about 100k measure taken at random intervals over a 10 year period. Each samples point is represented as a 3uple:

$$ (\text{id}, \text{date}, \text{weight}) $$

For some people, I have data over the entire time period. For others, only for a few years. Sometimes there is only a few days between two measurements for the same person. Other times, there might by years.

My initial idea was to interpolate the weight for each person at the end of each year based on the available records. But it was suggested to me interpolation would be a waste of time: using the last known value for people having measured their weight during the year should be accurate enough.

In my mind, using interpolation will somehow "smooth" the data curve for each individual, whereas using the last-known-value will be a stair step curve. But I can't figure out the implication of that in the whole population study. So:

Could you explain the pro and cons of interpolating missing data vs using the last-known-value when studying a large dataset?

I'm new to CrossValidated. I looked at the What topics can I ask about here? help page and I think this is the right place to ask. If it is not, please don't hesitate to point me to a more relevant SE site.

  • $\begingroup$ @Glen That was a typo--but I fixed the "dood" part immediately after having posted the question. It's curious you see that. But thank you for having pointed "enough" spelling error thought ;) $\endgroup$ – Sylvain Leroux Jan 23 at 12:52

You have repeated-measurement data, and you can model directly, using the actual dates. No need to do neither interpolation nor carry-forward. Using the R lme4 package for linear mixed models, you could try as a starting point something like

mod0 <- lme4::lmer(weight ~ date + (1 | id), data=your_data_frame, ...)

which is a model with random intercepts for each id. You could also try a term +date^2. This should be better than what you propose. Have a look at What are the differences between "Mixed Effects Modelling" and "Latent Growth Modelling"? and search for growth models.

The book Mixed-Effects Models in S and S-PLUS have examples that will get you started.

| cite | improve this answer | |
  • $\begingroup$ Thanks for the answer Kjetil. I'm very new to the field, so pardon me if I make few flimsy remarks here: (1) I read about repeated measure design, MEM and LGM -- and given I have missing data for some subjects at the start, end or in the middle of the experiment, isn't my case an instance of MEM rather than LME model? (2) Will your solution handle non-linear behaviors? More precisely, in my case, I suspect an exponential growth over some period up to some flat maximum, then followed by a more or less linear decline. ... $\endgroup$ – Sylvain Leroux Jan 22 at 14:08
  • 1
    $\begingroup$ ... (3) The book you mention has very good reviews but the price point is way above my budget. Any other reading suggestions? $\endgroup$ – Sylvain Leroux Jan 22 at 14:09
  • 1
    $\begingroup$ You can certainly include some modeling of non-linear behavior, search this site for growth models. Refs: There is a lot of information on the web, maybe bbolker.github.io/mixedmodels-misc/glmmFAQ.html, or better webcom.upmf-grenoble.fr/LIP/Perso/DMuller/M2R/R_et_Mixed/… $\endgroup$ – kjetil b halvorsen Jan 22 at 15:47
  • 1
    $\begingroup$ I find the second link you gave particularly accessible and instructive. Thanks for that Kjetil! $\endgroup$ – Sylvain Leroux Jan 23 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.