Multinomial classification where the response variable is monotonic over time

I have a set of feature and response variables measured for multiple subjects, each having multiple visit.

Let's define $X_t^p$ as the feature variable at the $t^{th}$ observation of subject $p$, $y_t^p$ as the response variable at the $t^{th}$ observation of subject $p$. The response variable takes values from an ordered class and I have multivariate features.

What I want to do is to find estimate all $y_t^p, t=1,2,3,..,n$ given $X_t^p, t=1,2,3,..,n$ for patient $p$

$$(X_1^p,X_2^p,X_3^p,..,X_n^p) \text{~} (y_1^p,y_2^p,y_3^p,..,y_n^p)....(1)$$

so that the estimations guarantee the following

$$\hat{y}_1^p\leq \hat{y}_2^p\leq \hat{y}_3^p\leq ..\leq \hat{y}_n^p....(2).$$ I checked ordered and isotonic classification, but they are not exactly what I am looking for because the final predictions are not guaranteed to be monotonic.

Also note that, the number of observations is not the same for all subjects. For example,

Subject 1
Obs.    X       y
1     (2,3)     2
2     (2,3)     2
3     (1,4)     3
4     (2,4)     3
5     (3,3)     4

Subject 2
Obs.    X       y
1     (0,0)     0
2     (0,1)     1
3     (1,0)     1
4     (0,0)     1
5     (1,1)     2
6     (2,1)     2
7     (2,3)     3
(data is not real)

I built my own (heuristic) approach which first assumes each observation is independent and using a classification model (e.g., multinomial logistic regression), it gets the probability predictions of all classes for each observation. Then, with the help of an integer programming model, it maximizes average probability of chosen classes for all observations for a subject satisfying (2). However, I am not sure if that is the most rigorous approach.

• Could you post a heatmap where the vertical and horizontal axes are from $X_t$ and the color intensity indicates the probability that $y_t > y_{t-1}$ given $X_t$? If this shows a strong trend, you could recode $y$ as increasing versus staying the same. – eric_kernfeld Nov 4 '17 at 12:30
• In fact, you could recode $y$ that way even if the plot looks useless. – eric_kernfeld Nov 4 '17 at 12:33
• (You could still include other predictors, such as the present state of $y$ or the history of $x$). – eric_kernfeld Nov 4 '17 at 12:42