Longitudinal panel data classification

Question

My problem context specifically lies in churn modeling, where accounts have account-specific attributes (like industry, number of employees, etc), but also have longitudinal yearly data (product usage data, contract premium costs, etc).

One example question we want to answer is: Given product usage data over time (annually), as well as their account-level characteristics, which accounts will churn the next year?

Are there any classification algorithms that can account for both time-series effects as well as account-level attributes? I feel like if I just run a normal classification algorithm (like RF, logistic regression, etc), then it'll assume every account is unique with its own attributes, without making the relationship that some observations are from the same account, and are related by time.

Also, if such a method exists, an interpretable one (where I can compute feature importance, etc) would be preferable. Thanks in advance!

Answer 1

Caveat: I am unfamiliar with churn models, but if they can fit into a linear regression form, you should be able to easily adapt this answer.

You can model both time effects and account effects using multilevel error-correction models with time of observation nested within accounts, and predictors $x$ existing at the time-in-account level (e.g., average transaction value) and $z$ in the account level (category of account a la corporate, personal, trust, etc.).

Error correction models come in different flavors, but a simple one based on a single-lag generating process which is capable of estimating instantaneous effects of change in predictors, short-run lagged effects of predictor levels, and long-run effects of lagged predictors on equilibrium for periods of observation $t$ nested within accounts $i$ is:

$$\Delta y_{ti} = \beta_{0ti} + \beta_{\text{c}}\left[y_{t-1i}-(x_{t-1i} + z_{t-1i})\right] + \beta_{\Delta x}\Delta x_{ti} + \beta_{x}x_{t-1i} \\ + \beta_{\Delta z}\Delta z_{ti} + \beta_{z}z_{t-1i} + \varepsilon_{ti} + \mu_{i}$$

Notes: The term in the square brackets may look scary, but it's just a new variable that is a function of the three terms shown. Also $\Delta y_{ti} = y_{ti}-y_{t-1i}, etc.$

You get the following effects through linear and nonlinear combinations of the estimates (for, say, $x$):

• Instantaneous effects of one-period change in $x$ on one-period change in $y$: $\beta_{\Delta x}$

• Short-run lagged effects of level of $x$ on one-period change in $y$: $\beta_{x} - \beta_{\text{c}} - \beta_{\Delta_{x}}$

• Long-run effects of lagged level of $x$ on equilibrium of $y$: $\frac{\beta_{\text{c}}-\beta_{x}}{\beta_{\text{c}}}$

Because this is a dynamic model (i.e. $y$ 'remembers' its past values), $\beta_{0}$ can't really be interpreted as an intercept, or as a constant linear trend, since it compounds each period (through $y_{t-1}$), but is going to be limited in effect if $|\beta_{\text{c}}|$ is close to 0, with increasingly lasting effects as $|\beta_{\text{c}}|$ approaches 1, and explosive effects if $|\beta_{\text{c}}|$ is greater than 1. Adding a random effect to $\beta_{0}$ permits this part of the equilibrium of $y$ to have a different dynamic for each account $i$.

For more see:

de Boef, S., & Keele, L. (2008). Taking Time Seriously. American Journal of Political Science, 52(1), 184–200.

Banerjee, A., Dolado, J. J., Galbraith, J. W., & Hendry, D. F. (1993). Co-integration, error correction, and the econometric analysis of non-stationary data. Oxford University Press, USA.

Answer 2

A couple of approaches come to mind:

• One simple way is run classifications for each category of account-specific features. If account-specific features are categorical already, train a classifier for each category. If they are continuous, you could bin them to obtain groups.

• Another way is to extract features from the time-series for each account, using e.g. tsfresh package, and then classify each account based on all features.

• A third approach would be to use Cox proportional hazard models.

• There are also more complicated models that may be applicable like point process models.

Answer 3

You can also consider adding account, the identifier, as a characteristic, this would be similar to a fixed effect regression.