Measuring longitudinal data where individuals have missing observations

Question

We have a longitudinal panel of X users with their online spending patterns and are trying to measure certain metrics within the panel. We have time series information about the users such as their total online spending, browsing habits, spending per online merchant etc. We also have cross sectional fixed data about the user like their geolocation, some demographic info etc. We are trying to look at a certain time series metric across the population of say X users.

Examples of what we are trying to measure

1. Total growth rate of spending month over month
2. Spend per transaction month over month
3. Spend per merchant per month
4. Other monthly (or period of choice) metrics

If all X users were reporting in the panel the entire time the exercise is easy, we simply calculate metric we need.

However only a small percent (15%) of the panel is reporting the entire length of the panel. Most users come into the panel late or drop out early. For each individual in the panel, the exact lifespan in the panel is fairly random. Moreover, some month, their usage is not complete, its partial and thus should not count, but this is a secondary concern.

The primary challenge is how to accurately calculate the metrics in question given this setup. One solution would be to construct a panel of users who are only present in the panel the entire length of the panel (lets call it Full Life User Panel). This would be a small % of users and assume that the users that are not in that panel behave in the same way as the users who are.

We are not aiming to measure the effect of one set of parameters on another. I.e. we are not trying to predict the spending at a specific merchant given information about the users’ other spending patterns and fixed attributed.

We can try to cluster the users using longitudinal clustering or some kind of latent growth curve analysis to “impute” the missing data. I haven’t found any landmark canonical material on this topic and would appreciate any help in addressing the question or references.

Answer 1

So many marketers assume that the goal of "single source" status at the household or individual level with respect to data is the only option. This can be a quite limiting, even procrustean mindset as it sets the bar unrealistically high given that 85% of your data is like swiss cheese wrt missing values. "Addressable" predictions can also be made at a higher levels of aggregation. It's a compromise, but it's a realistic one.

Here's one way to accomplish that: treat your data like a fractional factorial experimental design -- I don't mean this literally. FFD's are highly efficient and frugal approaches to obtaining experimentally derived estimates across a wide range of required information when faced with limited time and attention on the part of respondents. They achieve this by setting up a matrix structure where certain participants are exposed to some questions, but not others. It's only on the back-end, once the individual results are available for analysis, that the integration across respondents and questions is possible. Very simply, this involves creating a potentially large covariance matrix of the information relevant to whatever the analysis is that you want to do, bearing in mind that covariance matrices underlie most multivariate techniques anyway (i.e., this isn't a big technical leap). The trick is to be sure to turn off the "listwise deletion" options that are the default for many stats packages and routines. The analysis and model would unfold from that covariance matrix.

Since your data is longitudinal, we're also talking about creating transition matrices over time. Given that, my preference would be to use hidden markov model approaches. Steve Scott (Google) and Oded Netzer (Columbia) have written the most coherent papers on this class of models. In addition, there's a Chapman-Hall book out Hidden Markov and Other Models for Discrete- valued Time Series. Finally, the Latent Gold software has modules that will implement a markov chain for you.

Given this, imputing isn't required. Moreover, the complicated, messy and uncertain process of projecting from the 15% of complete information to the missing data isn't necessary. You only sacrifice a relatively small amount of specificity leveraging this covariance analysis.

Answer 2

It is a complex question, so just to get started: a reasonable certainty that values are missing at random (MAR) with respect to the variabels of interest is needed to justify imputation. It may be a hard case to prove that dropping out early is truly MAR. Moreover, whenever there are such low response rates (full panel), it is reasonable to assume they are not random, as you also note. However, you do not say how long the panel was - would it be possible to extract the information of interest ($spent/month) from partial participants? Also, "some month, their usage is not complete, its partial and thus should not count" actually looks like an issue amendable to imputation; although that depends on your question/concerns. Finally, approaches for imputation when involving longitudinal data http://cran.r-project.org/web/packages/longitudinalData/longitudinalData.pdf