Suppose I have a panel of explanatory variables $X_{it}$, for $i = 1 ... N$, $t = 1 ... T$, as well as a vector of binary outcome dependent variables $Y_{iT}$. So $Y$ is only observed at the final time $T$ and not at any earlier time. The fully general case is to have multiple $X_{ijt}$ for $j=1...K$ for each unit $i$ at each time $t$, but let's focus on the case $K=1$ for brevity.
Applications of such "unbalanced" $(X, Y)$ pairs with temporal correlated explanatory variables are e.g. (daily stock prices, quarterly dividends), (daily weather reports, yearly hurricanes) or (chess position features after each move, win/loss outcome at the end of the game).
I am interested in the (possibly non-linear) regression coefficients $\beta_t$ for doing prediction of $Y_{it}$, knowing that in the training data, given early observations of $X_{it}$ for $t < T$, it leads to final outcome $Y_{iT}$
$\hat{Y}_{it} = f(\sum_{k=1}^{t} X_{ik} \beta_k), \quad t = 1 ... T$
Coming from an econometrics background, I haven't seen much regression modelling applied to such data. OTOH, I have seen the following machine learning techniques being applied to such data:
- doing supervised learning on the entire data set, e.g. minimizing
$\sum_{i,t}\frac{1}{2}(Y_{it} - f(X_{it} \beta_t))^2$
by simply extrapolating/imputing the observed $Y$ to all previous points in time
$Y_{it} \equiv Y_{iT}, \quad t = 1... T-1$
This feels "wrong" because it will not take into account the temporal correlation between the different points in time.
- doing reinforcement learning such as temporal-difference with learning parameter $\alpha$ and discount parameter $\lambda$, and recursively solving for $\beta_t$ through back-propagation starting from $t=T$
$\Delta \beta_{t} = \alpha (\hat{Y}_{t+1} - \hat{Y}_{t}) \sum_{k=1}^{t} \lambda^{t-k} \nabla_{\beta} \hat{Y}_{k}$
with $\nabla_{\beta} \hat{Y}$ the gradient of $f()$ with respect to $\beta$.
This seems more "correct" because it takes the temporal structure into account, but the parameters $\alpha$ and $\lambda$ are kind of "ad hoc".
Question: is there literature on how to map the above supervised / reinforcement learning techniques into a regression framework as used in classical statistics / econometrics? In particular, I'd like to be able to estimate parameters $\beta_{t}$ in "one go" (i.e. for all $t=1...T$ simultaneously) by doing (non-linear) least-squares or maximum-likelihood on models such as
$Y_{iT} = f(\sum_{t=1}^T X_{it} \beta_{t}) + \epsilon_{i}$
I'd also be interested to learn whether the temporal difference learning meta-parameters $\alpha$ and $\lambda$ could be recovered from a maximum-likelihood formulation.