Probability of event happening when data is aggregated with many independent events over the course of time

Let's say you have $X$ coins, each with a differing probability of landing heads (e.g. coin 1 has 10% chance of landing heads, coin 2 has 20% chance of landing heads, etc.).

Now, let's say that you flip coin $i$ coin $Y_i$ times (each coin has a differing amount of flips). We know how many times each coin was flipped.

Now let's say you do this every day for a really long time and record that info. Example, with 2 coins, on day 1: 50 flips total, 30 heads total, coin1 was flipped 20 times and coin2 flipped 30 times. Day 2: 80 flips, 66 heads total, coin1 60 flips, coin2 20 flips.

What we know: total flips, total heads, how many times each coin was flipped.

Given that, is there a way to determine an approximate probability that the given coin will flip heads? In the above example, coin1 has a 100% probability of heads and coin2 has a 33% probability.

• p.s. new here, veteran of stackoverflow. I apologize in advance if formatting / wording was off. Aug 28, 2014 at 23:18
• Do we know how many heads each coin got?
– Dan
Aug 28, 2014 at 23:29
• That's what we're trying to determine Aug 28, 2014 at 23:31
• Do you know the probability of each coin beforehand?
– Dan
Aug 28, 2014 at 23:32
• No, all we know is the total amount of flips, total amount of heads, and how many times each coin was flipped Aug 28, 2014 at 23:32

Sorry for the delay, I was going to code this up as an example, but basically my method was going to agree with @Aniko's comment about using a linear regression.

So basically you can run a multiple linear regression where the outcome variable is number of heads and the predictors (or covariates) to the model is the number of tosses of each coin. Doing it so lets you model the scenario with as many coins as you want. Now, more formally, we want to model the following:

$$N_i=Y_{1i}p_1+Y_{2i}p_2+...Y_{Xi}p_X$$ where $N_i$ is the total number of heads on day $i$, $Y_{ji}$ is the $j^{th}$ coin thrown on day $i$, $X$ is the total number of coins, and $p_j$ is the probability of heads associated with the $j^{th}$ coin. So, we would like to be able to obtain estimates of the $p_j$'s.

One way (which is the way we shall illustrate) is to fit the above regression model using least squares. Then, the least square estimators will give the estimated probability of head for each coin (note we exclude the intercept).

To see this method in action, we will simulate data in R for 5,000 consecutive days with four coins each being flipped a random amount of times each day. Here is the R code for achieving this:

#The probabilities of heads for the 4 coins
p1 = 1
p2 = .3
p3 = .6
p4 = .45

#The number of days to flip the coins
days = 5000

#Variable for tracking number fo flips of each coin
coin1.flips = rep(NA,days)
coin2.flips = rep(NA,days)
coin3.flips = rep(NA,days)
coin4.flips = rep(NA,days)

#Variable for tracking total number of heads

#Variable for tracking total number of flips
flips = rep(NA,days)

for(i in 1:days){

#Sample a random number of flips for each coin
coin1.flips[i] = sample(10:100,1)
coin2.flips[i] = sample(10:100,1)
coin3.flips[i] = sample(10:100,1)
coin4.flips[i] = sample(10:100,1)

#Flip the coins
coin1 = rbinom(coin1.flips[i],1,p1)
coin2 = rbinom(coin2.flips[i],1,p2)
coin3 = rbinom(coin3.flips[i],1,p3)
coin4 = rbinom(coin4.flips[i],1,p4)

#Calculate total number of heads and flips
flips[i] = coin1.flips[i] + coin2.flips[i] + coin3.flips[i] + coin4.flips[i]

}

#Fit the least sqaures model

model


So we set the initial probabilities for the 4 coins to be $p_1 =1$, $p_2=0.3$, $p_3=0.6$, and $p_4=0.45$

Running the code we obtain the following estimates for the four probabilities:

> model

Call:
lm(formula = heads ~ coin1.flips + coin2.flips + coin3.flips +
coin4.flips - 1)

Coefficients:
coin1.flips  coin2.flips  coin3.flips  coin4.flips
1.0025       0.3022       0.6001       0.4459


And so we see that the coefficients (which are the estimated probabilities) are extremely close to the true solutions.

• I have two comments: First, your estimators are conditional on the matrix $\mathbf{Y}$. Second, note that the MoMs are not confined to the parameter space. Aug 29, 2014 at 19:44
• @Dennis, cool??
– Dan
Aug 29, 2014 at 20:10
• At Dan: Sure. I'm comfortable with making the inferences conditional on the sample sizes, but it's good to make that explicit. And this example nicely points up a problem with MoMs: they aren't necessarily confined to the parameter space if happens to be restricted. Aug 29, 2014 at 23:19

Starting with the case of two coins, we know that $$E\{T_k \}=m_k\theta_A+n_k\theta_B.$$ If $k=2$, we can solve $$t_1 = m_1\hat{\theta}_A +n_1\hat{\theta}_B$$ $$t_2 = m_2\hat{\theta}_A +n_2\hat{\theta}_B,$$ which is uniquely solvable if and only if $m_1n_2-m_2n_1 \neq 0$.

This yields Method of Moments (MoM) estimators. I don't see any guarantees that $\mathbf{\hat{\theta}}$ is in the unit square here. If you have fewer sets of tosses than coins, the system is underdetermined and infinitely many solutions exist. If you have more than two sets of tosses, the system is overdetermined, and a least squares solution can be derived using a generalized inverse.

For more than two coins, again you can get MoM estimators by equating the expected number of heads to their expectation. If you have as many sets of tosses as coins, a unique estimator will exist if the sample size matrix $\mathbf{N}$ is nonsingular. If there are more sets of tosses than coins, a least squares solution of the overdetermined system could be used.

• Just to clarify your "least squares solution" statement: one can use linear regression with number of heads in each experiment as the outcome variable, and the number of tosses of each coin as the predictors. The estimated coefficients will be the probability of heads for each coin. Aug 29, 2014 at 13:37
• That is one way to think of it. I was thinking in terms of solving an overdetermined system of equations rather than in terms of regression. I prefer to think of it in that fashion because it's not clear to me at this time that the whole panoply of regression statistics is useful in this application. I'm thinking particularly of the matrix $\mathbf{N}(\mathbf{N}'\mathbf{N})^{-1}\mathbf{N}'$ and the residual mean square. Aug 29, 2014 at 19:40