I need to segment a sequence of 0s and 1s by their proportion at relatively large scales. As an example, let's define 5 different states that represent 5 different ratios of 1s & 0s.
Alphabet: 1 and 0
State Definition emission prob.
state 0: 100% zeroes and 0% ones 0:0.999 1: 0.001
state 1: 75% zeroes and 25% ones 0:0.75 1: 0.25
state 2: 50% zeroes and 50% ones 0: 0.5 1: 0.5
state 3: 25% zeroes and 75% ones 0: 0.25 1: 0.25
state 4: 0% zeroes and 100% ones 0: 0.001 1: 0.999
Attempts:
With all the transition probabilities that I've tried so far and the emissions of each state, the output of my model sequences of states is only either state 0
or state 4
.
Example:
data (binary):
00000000000001111111111110000000000101010101010101010000001000100010011001000010
output I get no matter how I change the transition probs. (in states):
00000000000004444444444440000000000404040404040404040000004000400040044004000040
output I need (in states):
00000000000004444444444440000000000222222222222222220000001111111111111111111111
I have the impression that I am missing some basic theory rather than an implementation problem. For instance, I smoothed the data by aggregating obtaining the ratio of 1
vs 0
in a arbitrarily defined window, and in this way I can see the intermediate states between state 0
and state 4
. Nevertheless, I don't want to smooth the real data as I need to justify then the smoothing window size.
Is using HMMs a good solution for this problem?