I have a dataset with a bunch of entities (patients) and for each of these entities I have:

  1. A binary outcome specific to each entity (i.e. outcome does not vary in time)
  2. Some static predictors specific to each entity (e.g. gender, age)
  3. A single, time-varying measurement taken hourly for each entity, over some number of hours that is not necessarily the same for each (i.e. the time series for this measurement have different lengths)

I also suspect that the time-varying measurement has an effect on the outcome only when below a certain level. In other words, doing something as simple as just taking the mean measurement over all time points does not accomplish what I want.

What I would like to do would be to have a "change point" in the time-based measurement below which it's effect on the outcome can differ from its effect above that point, or in other words, I want two coefficients for that one predictor. I'm familiar with basic change point models but what I don't understand here is how I should literally structure my training data.

I don't want to do this, because it repeats the static covariates for each measurement:

Entity  Hour Gender  Age  Measurement  Outcome
1       1    Male    42   3.3          1
1       2    Male    42   8.9          1
1       3    Male    42   1.1          1
2       1    Female  33   2.3          0
2       2    Female  33   5.9          0

What other choices do I have then? What I'd really like is just one observation per entity but I'm not sure how to summarise the time series values into a single value for each when I also want the change point to be part of the estimation.

Does anybody have ideas on how to model something like this? Would it be unheard of to try to make the change point value part of the model itself and then use information criteria or cross-validation scores to determine where the best change point is? That could give me some sense of what change point is best but I'd love to keep that estimation all in one model if possible.

Confidence or credible intervals on the coefficients are a must and having the same for the change point would be nice too, though less necessary (and suggestions within the realm of R or python would be much appreciated).


P.S. Also, any recommendations on how to better understand the effects of repeated covariates on estimation would be a huge help too. I know that having repeated covariates mixed with non-repeated covariates is a bad thing, but maybe there are ways to adjust for the differences in true sample sizes? Mixed-effects regression would be great if it was applicable here, but I don't see how it is if the outcome does not also vary in time with each hourly measurement.

  • $\begingroup$ To me, this question is similar to work that's being done in econometrics which attempts to decompose group behavior as a function of the distribution of a time-varying predictor. Your case is similar in that, due to the static nature of the response for each patient, it can be treated as an aggregate, two-group problem with the "measurement" as the time varying predictor ... Decomposition Methods in Economics faculty.arts.ubc.ca/nfortin/w16045.pdf Generalize to your case as appropriate. $\endgroup$
    – user78229
    Nov 16, 2015 at 13:32
  • $\begingroup$ How could the time-varying measurement have any effect on outcome if outcome does not vary with time? Are you simply interested in estimating a smoothed proportion of prevalence of the outcome as a function of the continuous measurement? (Some context would really help elucidate this problem) $\endgroup$
    – AdamO
    Nov 17, 2015 at 12:44
  • $\begingroup$ Sure thing -- unfortunately I have to obfuscate the details a bit but let's say the outcome was whether or not a patient diagnosed with diabetes had kidney failure 5 years after diagnosis and the time-varying measurement is prior daily insulin level measurements. Above a certain value, the insulin levels should have no effect on the outcome but for patients who spend much time below that value, the outcome should be worse. In my case, I don't know what that value is but I'd like to estimate it and I'm ok with assuming a latent step function like that (but am open to nonlinear modeling too). $\endgroup$
    – Eric Czech
    Nov 17, 2015 at 13:40
  • $\begingroup$ Re: @DJohnson thanks for posting that paper, I had never heard of decomposition methods like that that attempt to account for differences in covariates as well .. very cool. I'm having a hard time seeing a generalization to my case since the change point is important but I can definitely see how it would apply if I was willing to work with quantiles or distribution statistics instead. That was one of the first roads I went down and I may fall back on it if there's no good way to more directly model the step function / change point. Let me know if I'm missing something in that paper though. $\endgroup$
    – Eric Czech
    Nov 17, 2015 at 13:56

1 Answer 1


One of the possible ideas would be to fit the following model:

$$P(Y_i = 1| z_i, \mu_{it}) = z_i\alpha + \beta\sum_{t=1}^T1\{\mu_{it}< \gamma\},$$

where $Y_i$ is the outcome for individual $i$, $z_i$ are the time constant additional variables and $\mu_{it}$ is the time-varying measurement. The parameters of such model would be $\alpha,\beta$ and cutt-off parameter $\gamma$. The precise form of the last term can be adjusted. Currently it measures how many periods the measurement was below the specified level and $\beta$ is interpreted as the effect of measurement being below the specified level.

Initially I would try to fit a range of values of prespecified $\gamma$, since it would not require any additional methods. If the results were favorable, i.e. $\beta$ being significant for a $\gamma$ being in some interval $(a,b)$ and non-significant for interval $(b,c)$, I would try to write down maximum likelihood function and fitting it numerically.

Update: I thought a bit how to make the step function smooth. First it is instructive to rewrite it as

$$S_i(\gamma)=\begin{cases}0, \text{if } \gamma<=\mu_{it}^{(1)},\\ j, \text{if } \mu_{it}^{(j)}<\gamma<=\mu_{it}^{(j+1)},\\ T, \text{if } \gamma\ge\mu_{it}^{(T)}.\end{cases}$$

Where $\mu_{it}^{(j)}$ $j=1,...,T$ are ordered values of $\mu_{it}$. We can divide by $T$ if we want, but that is not important. Having this function we can smooth it in various ways. Since we need derivatives, we cannot use linear approximation, but we can use splines (R implementation of splines helpfully provides derivatives too), or we can choose certain parametric function (various cdfs naturally come to mind) and simply fit it through the given points. The latter option would require investigating the data to see which cdf to choose. Or we can choose to expand MLE problem to fit a cdf too.

At first suppose that $S_i(\gamma)$ can be approximated by a single parametric cdf $F(\gamma|\theta)$, where $\theta$ is the parameter vector. If $F$ has a density $f$, the MLE for $\theta$ would be


We can make $\theta$ be a function of $z_i$ to loosen the hypothesis of a single $F$ for all $\mu_{it}$.

Naturally these thoughts are only conjectures, but I think both of these approaches can be made operational.

  • $\begingroup$ Thanks for the response! I like that idea and tried something similar with an mcmc sampler, marginalizing over a pre-specified grid of gamma values, so maybe I was on the right track there. I would love to come up with a ML solution but couldn't figure out how to compute a gradient wrt gamma. Do you know if that's even possible? I could probably go with a derivative free optimizer instead but another issue would be that I wouldn't know how to compute confidence intervals for gamma. Any ideas on that? Perhaps bootstrap sampling? Not sure if that would violate any implied assumptions. $\endgroup$
    – Eric Czech
    Nov 17, 2015 at 15:43
  • $\begingroup$ Use numerical gradients. If you pass only the ML to the numerical optimizer it will use the numerical gradients. If the optimizer achieves convergence you can use the inverse of numerical hessian as a covariance matrix, i.e. you can pretend that you have usual MLE problem. From mathematical point of view there are some problems here, but if everything works you can cross your fingers and pretend that you know nothing :) I would try however to smooth the step function, i.e. the sum of indicator functions. It looks like a lot like empirical cdf, so putting in some smooth cdf... $\endgroup$
    – mpiktas
    Nov 18, 2015 at 8:25
  • $\begingroup$ ... might solve the differentiability problem. $\endgroup$
    – mpiktas
    Nov 18, 2015 at 8:25
  • $\begingroup$ Ah nice, I've never heard of the hessian being used without true gradients but I guess that makes sense. I was trying to wrap my head around how I could include a continuous function for the step but not really getting anywhere -- do you know of any example functions that would make sense there? I'm assuming some kind of sigmoid curve would be best but then how would I use that? Would that last term become something like $\beta \sum_{t=1}^T f(\mu_{it}; \theta)$ where $\theta$ is the parameters of the sigmoid function being estimated (ie $f$)? $\endgroup$
    – Eric Czech
    Nov 18, 2015 at 10:54
  • $\begingroup$ Oh and if you have an idea for how to include a continuous function, I'll make this the answer for sure and edit your post to include that possibility (unless you'd prefer to do so). If anyone stumbles upon this post in the future, I think it would be nice to have both the discrete step function you initially proposed and a continuous version (which is helpful with something like Hamiltonian Monte Carlo that often requires an analytical gradient). $\endgroup$
    – Eric Czech
    Nov 18, 2015 at 11:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.