Are these trials identically distributed? These trials from a panel data are all Bernoulli experiments; do they have same probability of success p?

Question

Scenario: I have an ordered/indexed sample that supposedly comes from a binomial distribution with fixed probability of success p. (By ordered/indexed I mean that not only I care about the percentage of success out of the total n trials, but I care about the distribution of such successes or when they happen— so having 3 failures followed by 7 successes is NOT the same as having successes in the 1st, 5th, and 10th trials and failures in all the other 7 trials). Specific Scenario: Let’s say I performed 10 trials, which I assume come from a binomial distribution with fixed probability of success p. I got 3 successes in the first 3 trials followed by 7 failures in the last 7 trials.

Question: I could conclude that my population is distributed binomial with p=0.3 (the MLE based on the observation of the 10 trials). However, I don’t like that I got 3 successes followed by 7 failures (doesn’t look random at all). I am suspicious that, instead, the first three trials came from a binomial distribution with a high value for p, while the last 7 trials came from a binomial distribution with a low value for p. Null Hypothesis: p (the parameter for the binomial distribution) is constant throughout the whole experiment. Alternative Hypothesis: p actually changes at some point throughout the experiment (So I need a piecewise pdf to describe the experiment).

Initial thought: partition/dissect the data (which has thousands of observations, not just 10) into many subsections and see if success percentage (empirical value for p) is similar across all partitions. The empirical values for p obtained from each of the partitions should be distributed normally around the total percentage of success. I guess I could test for normality using some of the common techniques. (I wouldn’t make partitions too small that MLE would be poor, but I wouldn’t make them to large either that I wouldn’t be gaining much new info). This approach sounds a little rudimentary; I am wondering if there is a better/official way of doing this.

Answer 1

The operational way to frame IID Bernoulli data is given by De Finetti's "representation theorem", which is a famous result in probability (see related answers here, here, here and here). This theorem says that if you have a sequence of binary random variables, they will be IID if they are exchangeable. Exchangeability means that for any finite number of outcomes in the sequence, if you were to rearrange the outcomes then the probability of that overall result would not change (i.e., the order of the values does not affect the overall probability of a particular set of outcomes).

In your question you note that you are interested in the order of the outcomes in your analysis. The relevant question is, for your data, can the order of the outcomes plausibly affect the probability of a particular set of outcomes? If it doesn't affect the overall probability then you have exchangeability, which means that you are correct to assume that your values are IID Bernoulli random variables with a fixed occurrence probability. If it does then you do not have IID data and the IID Bernoulli model is inappropriate.

Sometimes you can answer this question by looking at the underlying mechanism generated the data (if it is real data) and seeing if there is any plausible way that the order of a set of outcomes could affect the overall probability. In cases where this is not possible, you can test exchangeability empirically using various kinds of runs tests (see related answers here, here and here). The latter is what I would suggest in your case --- take your data and perform a runs test to see if there is evidence of a breach of exchangeability. If so, this means that your IID Bernoulli model is probably not a reasonable model and you would then need to examine alternatives that build in some depenence between outcomes and/or changing marginal probabilities of outcomes.