Mixture model fixed effects

Question

One standard model used with panel data is fixed effects: $y_{it} = \mu_i + \theta_t + \epsilon_{it}$, where $i$ is the individual and $t$ is time subscripts. This can be estimated easily with OLS and dummy variables.

The model assumes that there is a single underlying time series, $\theta_t$ for $t = 1,2,...,T$. All individuals in the data are assumed to follow this time series plus some individual effect, which is constant relative to time.

Suppose, however, that there are two or more groups of individuals, that each group has its own time series, and that, before looking at the data, we don't know who is in which group. I would like to estimate the fixed effects model in this case and to figure out which individual is in which group / cluster.

The approach that I thought of is that the model should become: $y_{it} = \mu_i + \sum_j \pi_{ij} \theta_{jt} + \epsilon_{it}$. $j$ indicates the group $1,2,...,J$. $\pi_{ij}$ is the probability that individual $i$ is in group $j$. $\theta_{jt}$ is the time effect for group $j$ at time $t$.

In my particular application, $\{\pi_{ij}\}$ needs to be a set of additional parameters. However, I can see that in other applications, it could be modeled as a function of some covariates.

Is this model a good approach? Has anyone tried it before? How do I estimate this model? The model has a lot of parameters. I've tried basic optimization, and it has not worked. Ideally, I am looking for software to do the estimation, such as a package in R; or, a solid reference that I could use to program this.

If the model that I've proposed is not a good approach, how else could I solve this problem? One possibility that I could see is first figuring out which individual belongs to which cluster, and then estimating a regular fixed effects model on each cluster. The issue then is how to perform the data classification. Regardless of the approach, I am still looking either for software or a good reference.

Answer 1

This type of model has been entertained by education researchers for years under the name of growth mixture model (do different students exhibit different rate of learning?), although they work with it as a random effects model. I don't think you'd be able to come up with a proper fixed effect estimation for this model, as it may lack the sufficient statistic that you could condition on. But a full blown ML model should not be extremely difficult to fit.

In Stata, you could try fmm with a full set of panel id dummies and/or interactions with time, although this is a very wasteful approach in terms of degrees of freedom.

Answer 2

You are trying to fit a "mixture model", though I would suggest not having a separate set of $\pi_{ij}$ parameters for every person, but rather model these probabilities as a function (i.e. logistic) of some covariates. You could even start with a constant $\pi_j$. The typical approach for fitting mixture models is the EM (expectation-maximization) algorithm, though for the simpler versions even direct maximization would work.

Perhaps searching on these terms will help you find a solution that already exists.

Answer 3

I suspect you actually mean $y_{it}=μ_i+∑_j I_{ij}θ_{jt}+ϵ_{it}$ where $I_{ij}$ is an indicator for individual $i$ belonging to group $j$ (with probability $\pi_{ij}$. I also wonder if you really want to allow for different individuals to have different group assignment probabilities? I suspect $\pi_j$ will suffice?

I guess it could be seen as a random/mixed effect (not a mixture) with a multinomial effect. This might be an improper term since "random effects" is a typically reseved for Gaussian random effects.

Also note that you are not dealing with a Gaussian mixture, because you allow an individuals shift of the Gaussian ($\mu_i$).

Anyhow, I can think of two approaches:

1. Write the likelihood and maximize it using en EM algorithm which you can write yourself.
2. Compute $\mu_i$ by averaging within individuals (assuming you have enough observations). The residuals will be distributed like a Gaussian location mixture at each $t$. You can use a standard EM implementation such as mixtools R package to estimate $\{\theta_{jt}\}$ and $\{\pi_j\}$.