If I understand you correctly, you have two experiments, let's call them $X$ and $Y$, where at $i$-th timepoint two groups of size $n_X$ and $n_Y$ (here they are always the same for simplicity, but they can be differ at different timepoints) are assigned to two conditions and you observe $x_i$ and $y_i$ successes in both groups. So the probability of success group $X$ can be seen as a binomial random variable with $n_X$ number of trials and $\pi_X$ probability of success, that is unknown to us (same applies to $Y$).
This problem can be approached from the Bayesian perspective, using beta-binomial model, where the posterior distribution for the probability of success (by conjugacy) is a beta distribution
$$
\pi_X|x_i \sim \mathcal{B}(\alpha_X + x_i, \;\beta_X + n_X - x_i)
$$
where $\alpha_X,\beta_X$ are the parameters from the $\mathcal{B}(\alpha_X, \beta_X)$ prior distribution for $\pi_X$ (say, uniform $\alpha_X=\beta_X = 1$). If you want to update what you learned at $i$-th point, given the results at $i+1$ point, then the posterior distribution becomes
$$
\pi_X|x_i,x_{i+1} \sim \mathcal{B}(\alpha_X + x_i + x_{i+1}, \;\beta_X + 2n_X - x_i - x_{i+1})
$$
etc.
From this, you can easily obtain point estimates for $\pi_X$ (e.g. posterior mean, median, or mode) and credible intervals (e.g. taking the appropriate quantiles from the posterior distribution). You can even track the changes over time and produce a plot like the one below (here using simulated data).
Instead of plotting the individual distributions of $\pi_X$ and $\pi_Y$, you can instead plot the distribution of their differences $d = \pi_X - \pi_Y$. The distribution can be easily obtained by drawing random samples from posterior distributions of $\pi_X$ and $\pi_Y$ and then taking their differences, the empirical distribution of those differences can be used as an estimate of the posterior distribution.
Alternatively, you could look directly at the posterior predictive distributions of $X$ and $Y$, by simulating the draws from the beta-binomial distributions with appropriate parameters and then counting how often $x>y$ among the simulated values (so probability is count divided by the number of random draws).
All this, obviously, operates on the "cumulative" data, since after each new experiment you update you knowledge. That is why the intervals get narrower over time. As you asked, this gives you the intervals and enables to compare the changes in your knowledge about the parameters over time. Notice however, that you seemed to be asking about comparing the distributions at some arbitrary time window, but this approach would also take into consideration your past knowledge (what seems reasonable!).
If you assume that during some periods of the experiments there were external factors that could influence the results and this is the reason that you want to compare those periods, then you can do better then this. Instead of looking at the plots and making informal interpretations, you could use more complicated Bayesian model that includes additional covariates (e.g. weather). For this, you would define the model in terms of Bayesian logistic regression model, but this is a different story.
Since sometimes code is worth more then thousand words, I attach the R code that was used to produce the examples.
library(ggplot2)
set.seed(123)
# simulated data
n <- 100
nx <- ny <- 500
x <- rbinom(n, nx, 0.22)
y <- rbinom(n, ny, 0.21)
plot(x,type = "l", ylim = c(80, 140))
lines(y, col = "gray")
# posterior parameters for pi_X
post_x_a <- 1 + cumsum(x)
post_x_b <- 1 + cumsum(nx - x)
# posterior parameters for pi_Y
post_y_a <- 1 + cumsum(y)
post_y_b <- 1 + cumsum(ny - y)
post_df <- data.frame(
# posterior mean and quantiles for pi_X
x_mu = post_x_a / (post_x_a + post_x_b),
x_ymin = qbeta(0.025, post_x_a, post_x_b),
x_ymax = qbeta(0.975, post_x_a, post_x_b),
# posterior mean and quantiles for pi_Y
y_mu = post_y_a / (post_y_a + post_y_b),
y_ymin = qbeta(0.025, post_y_a, post_y_b),
y_ymax = qbeta(0.975, post_y_a, post_y_b)
)
ggplot(aes(y = x_mu, x = 1:n), data = post_df) +
geom_line(color = "blue") +
geom_ribbon(aes(
ymin = x_ymin,
ymax = x_ymax
), alpha = 0.3, fill = "lightblue") +
geom_line(aes(y = y_mu, x = 1:n), color = "red") +
geom_ribbon(aes(
ymin = y_ymin,
ymax = y_ymax
), alpha = 0.3, fill = "red") +
xlab("") + ylab(expression(list(pi[X], pi[Y]))) +
theme_minimal()
# m=50000 random draws from the posterior distribution
m <- 50000
post_x_sim <- matrix(rbeta(n*m, post_x_a, post_x_b), nrow = m, byrow = TRUE)
post_y_sim <- matrix(rbeta(n*m, post_y_a, post_y_b), nrow = m, byrow = TRUE)
# posterior distribution of d = pi_X - pi_y
post_xy_diff <- post_x_sim - post_y_sim
post_sim_df <- data.frame(
# posterior mean
diff_mu = colMeans(post_xy_diff),
# posterior quantiles
diff_ymin = apply(post_xy_diff, 2, function(x) quantile(x, probs = 0.025)),
diff_ymax = apply(post_xy_diff, 2, function(x) quantile(x, probs = 0.975))
)
ggplot(aes(y = diff_mu, x = 1:n), data = post_sim_df) +
geom_line(color = "darkgray") +
geom_ribbon(aes(
ymin = diff_ymin,
ymax = diff_ymax
), alpha = 0.3, fill = "lightgray") +
xlab("") + ylab(expression(d == pi[X] - pi[Y])) +
theme_minimal()
library(extraDistr)
m <- 50000
post_x_sim <- matrix(rbbinom(n*m, n, post_x_a, post_x_b), nrow = m, byrow = TRUE)
post_y_sim <- matrix(rbbinom(n*m, n, post_y_a, post_y_b), nrow = m, byrow = TRUE)
post_xy_diff <- post_x_sim - post_y_sim
post_sim_df <- data.frame(
# posterior mean
diff_mu = colMeans(post_xy_diff),
# posterior quantiles
diff_ymin = apply(post_xy_diff, 2, function(x) quantile(x, probs = 0.025)),
diff_ymax = apply(post_xy_diff, 2, function(x) quantile(x, probs = 0.975)),
# posterior probability of pi_X > pi_Y
diff_prob = apply(post_xy_diff, 2, function(x) mean(x > 0))
)
ggplot(aes(y = diff_prob, x = 1:n), data = post_sim_df) +
geom_line(color = "darkgray") +
xlab("") + ylab(expression(Pr(X > Y))) +
theme_minimal()
