Say we've a time indexed sequence of feature vectors/covariates/independent variables $x_t$ at time $t$. Say also we've a corresponding time indexed sequence of variates/dependent variables $y_t$. Now normally, when we solve classification problems, we assume that the covariates/features form a random sample, i.e. $x_i$'s are values of the random variables $X_i$'s, and that $X_i$'s are iid random variables. Clearly that assumption doesn't necessarily hold for time indexed covariates. So my question is: how to go about solving a classification problem where independent variables/covariates/feature vectors form a time series?
Now a direct aproaches that comes to mind is: Come up with a time dependent classification model (e.g. logistic regression) of the form: $y_t|x_t \sim Ber (<\theta_t, x_t>)$ for time variable $t$. Then we can try to construct the joint probability density $f(y_1, ...y_n|x_1, ... x_n)$ and maximize it w.r.t. $\theta_t$ and then try to learn about the function $\theta_t$. But we face the problem that unlike standard logistic regression we can NOT write $f(y_1, ...y_n|x_1, ... x_n)= \Pi_{t=1}^{n} f_t(y_t|x_t)$, as the the distribution of $y_t|x_t$ may depend on that of $y_s|x_s$ when $s < t$. Now of course, we can use Markov chain or hidden markov models to assume that $y_t|x_t$ depnds only on previous $d$ "events": $y_s|x_s, t-d+1 \leq s \leq t$. But all time series may not satisfy this assumtion. So how to get around that?
Is there a concre mathematical theory of time dependent classification models?