Random coef models, applied to longitudinal data, capture response heterogeneity by cross-sectional unit.

I've got a longitudinal prediction problem, in which I know that some "features" (or derived features) should be treated like "fixed effects" -- their influence is homogeneous across cross-sectional units, while others should be treated like "random effects" -- their influence should vary across CSUs.

I've tried to implement this in a couple of ways, but I'm running into trouble, largely stemming from the fact that RNNs represent time series as a N x t x P array, and a random coef would be represented by a P x k x N coefficient matrix multiplied by a N*t x P longitudinal dataset.

So, my problems are both conceptual and implementation-related. I don't know how linear algebra works beyond two dimensions, nor do I know how to implement it in Keras (or anywhere else, for that matter). I'd appreciate pointers to approaches that others have taken to these sorts of problems, or suggestions for fundamental things to read.

  • 1
    $\begingroup$ I think you would like to call them random effects models or mixed effects models (the hierarchical model term is probably redundant for what is asked). Random coefficient models is not a standard term. (+1 though, thoughtful question obviously) $\endgroup$ – usεr11852 says Reinstate Monic Apr 5 '19 at 17:08
  • 1
    $\begingroup$ Pretty hot topic currently, but I'm afraid that it's still unsolved. Bayesian neural networks seem to be a way to go. $\endgroup$ – Tim Apr 5 '19 at 17:08
  • 1
    $\begingroup$ @Tim Maybe within something like Edward? $\endgroup$ – usεr11852 says Reinstate Monic Apr 5 '19 at 17:11
  • $\begingroup$ I don't know whether to be relieved or annoyed that my problem is at the research frontier. @usεr11852 If one simply wants random intercepts, Keras can handle a 2D matrix of CSU dummies with aplomb, L2-penalized if you'd like them to be random rather than fixed. "random coefficients" is the standard term in econometrics when the random term is multiplied by a non-dummy variable. (I originally learned stats in an economics department). But it might confuse people, so I'll edit the title. $\endgroup$ – generic_user Apr 5 '19 at 17:24
  • 1
    $\begingroup$ @generic_user: Probably you should interpret it as a very good thing. No fun in asking obvious questions! ☺ (I suspected an econometrics connection but then again that perplexes things further as random effects in Econometrics and Biostatistics are referring to exactly the same thing.) $\endgroup$ – usεr11852 says Reinstate Monic Apr 5 '19 at 17:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.