Variance in a binomial sequence, when successes are bunched

Question

Imagine a randomly generated binary series, e.g.:

0011001111110101001.

Now imagine that it's not entirely random, but that the 1s and 0s tend to come roughly in bunches, e.g.:

0000011111000000000000011111100000110001111111100111110000010000111111111111111111000000000000011111 (100 digits).

I want to measure the ratio of 1s and 0s. In the above sequence, it happens to be 1:1. There are 50 0s an 50 1s.

But imagine I can only sample a part of the sequence, say 10 consecutive digits. If I happened to select a bunch of 10 zeros together, I would have to conclude that the sequence is 100% zeros. Conversely If I happened to select a bunch of 10 ones together, I would have to conclude that the sequence is 100% ones. Usually, my 10 consecutive digits will give me some intermediate fraction.

To take it to the extreme, if I had 50 x 0 followed by 50 x 1, and I was sampling 10 consecutive numbers, I would almost always find 100% 0 or 100% 1 and so the variance is wide.

If the numbers are more like 0010011110111101010101.... (i.e. bunches of 2 or 3) then the ratio is going to be much more like 50:50 each time you take a sample.

I'm trying to figure out how I can quantify the uncertainty in my 0s:1s ratio. I know that if the numbers were purely random, I could just use binomial statistics to calculate the standard deviation of number of successes when choosing 10 numbers (sqrt(2.5) I believe).

But I think the fact that my numbers are more likely to bunch together complicates things. Is there a way I can take into account how long my "bunches" usually are and how many numbers I am sampling, to say something about the variance of the fraction of 0s/1s I will measure.

Answer 1

If your pattern repeats exactly (as you commented), then we are dealing with finite population and it is quite easy to calculate probabilities. Is such case the only thing you need to do is to enumerate all the possibilities and count the cases where proportion of $1$'s is exactly $50\%$. For example, while using subsequences of length $10$ probability of observing five $1$'s is $0.27$.

n <- length(x) # x the sequence coded as numeric vector
k <- 10

# using modulo since the sequence repeats itself
countOnes <- function(i) sum(x[(((i:(i+k-1))-1) %% n)+1])
allSeq <- vapply(1:n, countOnes, numeric(1))

that returns

> summary(allSeq)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
      0       3       5       5       7      10
> mean(allSeq == round(k/2))
[1] 0.27

However if your sequence does not repeat exactly, than this solution is only as valid as representative is your sequence for the whole population. Moreover, in such case different methods may be more appropriate (e.g. time-series models, or treating your sequence as Markov chain - as described by mef, etc.).

Answer 2

You can think of this as a two-state Markov chain. There are four possible sequences of length two: "00", "01", "10", and "11". Let $q_0$ denote the probability of "01" and let $q_1$ denote the probability of "10". (Then the probability of "00" is $1-q_0$ and the probability of "11" is $1-q_1$.) Given these conditional probabilities, the unconditional probability of "0" is $r_0 = q_1/(q_0+q_1)$. If $q_0 = q_1$, then $r_0 = \frac12$. If $q_0 = 1-q_1$, then there is no state dependence. The only way both can hold is for $q_0 = q_1 = \frac12$.

Given your example of 100 digits, there are 99 sequences of length two. There are 42 "00", 8 "01", 7 "10", and 42 "11". Even though there are an equal number of 0's and 1's, there are not an equal number of transitions: there's one more "01" than "10". The likelihood for $q_0$ is $q_0^{8}\,(1-q_0)^{42}$ and the likelihood for $q_1$ is $q_1^{7}\,(1-q_1)^{42}$.

I take a Bayesian approach to inference. Because I have no special knowledge about $q_0$ or $q_1$, I put uniform priors on both $q_0$ and $q_1$ over the unit interval. (Knowledge you have about $q_0$ and $q_1$ before you see the data can be incorporated into their prior distributions.) Then the posterior distributions are $$ q_0|y \sim \textsf{Beta}(9, 43) $$ and $$ q_1|y \sim \textsf{Beta}(8, 43) $$ where $y$ denotes your data. Now it's easy to make draws of $r_0$ from the posterior. Simply draw $q_0$ and $q_1$ from their respective posteriors and compute $r_0$ from those draws. Do this many times. These draws of $r_0$ provide an approximation to its posterior distribution.

I took $10^6$ draws and got a mean of 0.475 and a standard deviation of 0.109. Also, 90% of the probability was between 0.296 and 0.655. Here's a histogram:

Suppose instead your data consisted of a sequence of 10 zeros. Then we have $$ q_0|y \sim \textsf{Beta}(1, 10) $$ and $$ q_1|y \sim \textsf{Beta}(1, 1) $$ The mean is 0.802 and 90% of the probability is above 0.523. The histogram looks like this:

So this approach takes into account the concerns you have expressed.

Edit

Thinking about this some more, I realized that I should take into account the "unconditional" probability of the first item in the sequence. In the example given by the OP, the first item is "0", the unconditional probability of which is $q_1/(q_0+q_1)$. Multiplying this times the likelihoods for the transitions produces the fully symmetric likelihood: $$ \frac{q_0^{8}\,(1-q_0)^{42}\,q_1^{8}\,(1-q_1)^{42}}{q_0+q_1}. $$ There is a dependence now due to the denominator, so $q_0$ and $q_1$ must be drawn jointly from the posterior. With a flat prior over the unit square, the posterior is proportional to the likelihood. Draws can be made from the posterior using the accept-reject method. (There may be more efficient ways, but this works.) I took $10^5$ draws. The mean is 0.500 and the standard deviation is 0.106. (The histogram looks pretty much the same as above.)

For the case of 10 zeros, we can make the same adjustment to the likelihood, producing $$ \frac{(1-q_0)^{9}\,q_1}{q_0+q_1}. $$ In this case, the posterior mean is 0.852 and 90% of the probability is above 0.66. Again, the histogram looks very similar to the one above.