My problem is similar to Markov Discrete Process, with one little but - it doesn't have true markov property. a probability to move to next state rarely depend only on current state but rather on 3-10 previous states.. these numbers (3-10) are not known and are a guess.
I want an idea on how to approach this using a deep learning machine (RNN perhaps) which will discover most robust patterns (most statistically stable dependencies between the current state and previous states), one state can be part of many patterns simultaneosly.
Then I want the machine to predict next state as a matrix of probabilities for each possible values, that is if there are 5 possible values for each states then I need 5 prob numbers for one predicted state. Similar to markov's transition matrix.
how would you approach this problem from a DL prospective?
P.S. it appears to be a case of continuous unsupervised learning