1
$\begingroup$

I have a process that can be modeled by looking at the occurrence of particular patterns in a 12 digit binary string to determine if those patterns occur more or less often than they would randomly. I am assuming that each digit has an equal chance of being 0 or 1, but throwing out the case of 00_00_00_00_00_00 since that pattern has a very high likelihood of occurring (>50%) and I'm only interested in the cases where there is a 1 somewhere in the binary string. The total number of possibilities is then 2^12 - 1 = 4095. The patterns I'm interested in are any of the following:

.1_1._.._.._.._..
.._.1_1._.._.._..
.._.._.1_1._.._..
.._.._.._.1_1._..
.._.._.._.._.1_1.

Where . can be any other digit. This means that the expected probability of each of these is 2^10 / (2^12 - 1), which is approximately 25.0006%. However, since there are 5 possible patterns simply summing the probabilities yields a value greater than 100%.

I have experimental data that I will be calculating the actual observations from to compare to, I want to know if any of those patterns is different than what random chance would suggest. My first thought is to use a chi-squared test with the null hypothesis being that there is no difference and the alternate hypothesis being that there is a difference. I think the degrees of freedom would be 4, but I'm not sure since a single observation can count for multiple patterns that I'm interested in.

Is chi-squared the correct method to use here? If so, would my degrees of freedom be 4?

(Note: I organized the binary string into pairs of digits because in the physical process I'm measuring that's how they're organized. There's a specific failure case I'm trying to analyze where two adjacent pairs might be experiencing some kind of interaction, but only on their adjacent digits.)

$\endgroup$
  • $\begingroup$ Is it safe to consider that, for a given string, the pairs you are investigating (._.)(no smiley intended) are independent from each others ? If so, does the pattern ".1_11_1[..]" contribute more to your scientific claim than let's say ".0_01_1[..]" or ".1_10_0[..]" alone ? Just to be sure, do you assume the equiprobability of 0 and 1 for the sake of statistical testing or because you have strong reasons to think it's a real property of your data ? $\endgroup$ – brumar Jun 21 '16 at 14:58
  • $\begingroup$ @brumar Each case of 1_1 should be independent, but in reality it probably has some dependence. I'm considering it a simplification for the sake of sanity. The likelihood of a 0 or 1 is not 50%, but the likelihood of any individual digit being 1 is the same, so the first digit has the same probability p of being 1 as the second digit, and so on. $\endgroup$ – bheklilr Jun 21 '16 at 18:40
  • 1
    $\begingroup$ Ok, that's way more clear to me now and I would be able to suggest an answer soon . As a side note, your estimation of $0.25$ can be very wrong if $p$ is different from $0.5$. Enumerating cases work only if your cases are equiprobable. Tell me how do you feel about this simple reformulation. You want to test $H0 : P(1|1)=P(1|0)$ against $H1 : P(1|1)>P(1|0)$ where $P(y|x)$ is $P(y)$ given the fact that $x$ has been observed in the first item of the studied pair ($x$_$y$). $\endgroup$ – brumar Jun 21 '16 at 19:29
  • $\begingroup$ I am also wondering about the frequency of the 00_[...]_00 chain. Is it just some kind of extra noise, or is it caused by to the high frequency of 0? $\endgroup$ – brumar Jun 21 '16 at 19:41
  • $\begingroup$ @brumar The all $0$ chain is actually the success case. I'm trying to study a process where there are 12 measurements being made simultaneously. These measurements are ordered in the same way in the binary string as they are physically. The current theory is that there is a flaw in the machine that causes it to sometimes skew the measurement for the two measurements on either side of a $\_$. It isn't an easy measurement to characterize by a single value, but we can characterize it into "good" and "bad". Here, I've chosen 0 to represent "good". Ideally, all 0s would be a very common case. $\endgroup$ – bheklilr Jun 21 '16 at 20:09
1
$\begingroup$

Testing independence

After our discussion in the comment space, I think testing independancy using chi-squared is straightforward. You just want to test the indepedance of X and Y where Y follows X on the adjacent pair. First fill this table with these occurences computed accross your whole dataset.

            X
            0   1
      Y 0   a   b
        1   c   d

The columns specify the first part of your pair, and the the rows the second part. To be concrete :

  • $a$ is the number of times you got $0$ _ $0$
  • $b$ is the number of times you got $1$ _ $0$
  • $c$ is the number of times you got $0$ _ $1$
  • $d$ is the number of times you got $1$ _ $1$

A chi-squared test can be done in one line of code. It has only one degree of freedom. In R : chisq.test(cbind(c(a,b),c(c,d)))

Comparing proportions

I wrote the previous passage because you were asking about chi-squared test. I don't think this is the most relevant test here. The problem with this approach is that it does not take into account that your alternative hypothesis is $P(1|1)>P(1|0)$ Indeed, a chisq test could reject independence if the contrary was true ($P(1|1)<P(1|0)$). If your dataset is large enough I would suggest to compare proportions instead. This approach is well described here. I'll adapt it for an 1-sided test here to give the p.value needed.

z.prop = function(a,b,c,d){ numerator = (d/(d+b))-(c/(a+c)) p.common = (c+d) / (a+b+c+d) denominator = sqrt(p.common * (1-p.common) * (1/(a+c) + 1/(d+b))) z.prop.ris = numerator / denominator return(pnorm(z.prop.ris,lower.tail = FALSE)) } z.prop(a,b,c,d)

lower.tail = FALSE allows you to look only on the "right side" of the normal distribution.

$\endgroup$
  • $\begingroup$ I wanted to let you know that I'm not ignoring your response and I appreciate the help, I've been out sick the last couple days and haven't been able to get around to implementing your answer to see if it works for me. It might be a few more days due to the weekend, but I'll be coming back to this because I think it'll be quite helpful. $\endgroup$ – bheklilr Jun 24 '16 at 14:38
  • $\begingroup$ No problem. I wish you a good recovery. $\endgroup$ – brumar Jun 24 '16 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.