# Are these trials identically distributed? These trials from a panel data are all Bernoulli experiments; do they have same probability of success p?

Scenario: I have an ordered/indexed sample that supposedly comes from a binomial distribution with fixed probability of success p. (By ordered/indexed I mean that not only I care about the percentage of success out of the total n trials, but I care about the distribution of such successes or when they happen— so having 3 failures followed by 7 successes is NOT the same as having successes in the 1st, 5th, and 10th trials and failures in all the other 7 trials). Specific Scenario: Let’s say I performed 10 trials, which I assume come from a binomial distribution with fixed probability of success p. I got 3 successes in the first 3 trials followed by 7 failures in the last 7 trials.

Question: I could conclude that my population is distributed binomial with p=0.3 (the MLE based on the observation of the 10 trials). However, I don’t like that I got 3 successes followed by 7 failures (doesn’t look random at all). I am suspicious that, instead, the first three trials came from a binomial distribution with a high value for p, while the last 7 trials came from a binomial distribution with a low value for p. Null Hypothesis: p (the parameter for the binomial distribution) is constant throughout the whole experiment. Alternative Hypothesis: p actually changes at some point throughout the experiment (So I need a piecewise pdf to describe the experiment).

Initial thought: partition/dissect the data (which has thousands of observations, not just 10) into many subsections and see if success percentage (empirical value for p) is similar across all partitions. The empirical values for p obtained from each of the partitions should be distributed normally around the total percentage of success. I guess I could test for normality using some of the common techniques. (I wouldn’t make partitions too small that MLE would be poor, but I wouldn’t make them to large either that I wouldn’t be gaining much new info). This approach sounds a little rudimentary; I am wondering if there is a better/official way of doing this.

• Hi @Zmod2Z, welcome to CV. I'm curious about the "ordered" nature of your data. Binomial data by definition is made up of independent Bernoulli trials, so the ordering doesn't matter. I am unconvinced that, if one trial depends on another, that Binomial is an appropriate model. Jun 7 at 1:05
• If you suspected a trend in $p$ across trials before seeing the data you could test for that. If you expected a jump change in $p$ at some particular trial, you could test for that, and so on. However, it appears your suspicions about changing $p$ were aroused by seeing the very data you want to test the hypothesis on. This plays havoc with significance levels ($\alpha$) and consequently, the behaviour of p-values. They don't have the properties you wish them to have. For example, see en.wikipedia.org/wiki/Testing_hypotheses_suggested_by_the_data . Even without a test .... ctd Jun 7 at 2:37
• ctd ... the properties of estimates (and intervals, etc) are similarly impacted by using the same data to choose/formulate a model (Bernoulli trials with changing $p$) and estimate it. Jun 7 at 2:41