I asked this question at math.stackexchange.com first, but nobody answered. Perhaps statisticians can help me.
The question is like this:
I have a signal generator, which each second generates one of the letters 'a', 'b' or 'c'.
I don't know anything else about this signal generator, but I suspect that there are some patterns to it.
The signal generator is started at time 0. Given output of the signal from time 0 to time n, I need to forecast its output at time n+1.
I suspect that there are some patterns to the signal, so I create a frequency table:
For each sub-string (up to length 7) of the first n symbols of the signal generated data, I calculate three values: the number of occurrences of 'a','b', and 'c' after that sub-string.
So, for example for sub-string "abc"(if it exists in the data), I store:
- the number of cases when symbol 'a' comes after string "abc"
- the number of cases when symbol 'b' comes after string "abc"
- the number of cases when symbol 'c' comes after string "abc"
Now that I have all these data, I have 7 predictions as to what the next symbol could be. If for example the last 7 characters of the signal are "acbccba", then:
If I look just at the one-character frequency table, then I will have a certain prediction, which would look like: "Since the last character of the string is 'a', and since coming directly after character 'a' there were 40 cases of letter 'a', 25 cases of letter 'b', and 130 cases of letter 'c', I predict that the next character will be letter 'c'"
Similarly for the last 2-letter ("ba"), 3-letter ("cba"), ... , 7-letter ("acbccba"). So in the end I have 7 predictions.
The question is, how do I find which next character is actually the most probable for this signal generator? Different predictions are based on different sample sizes, so how do I combine them effectively?