Concentration bounds on a sequence of (0,1)-experiments I have a $(0,1)$-experiment that returns $1$ with probability $p$ and $0$
with probability $1-p$. Let $X_i$ be the random variable describing
the outcome of iteration $i$ of the experiment. I'm interested in the
value $Y_t=\sum_{i=0}^t X_i$, which counts the number of successes
since the beginning of the sequence. The sequence terminates as soon
as $Y_t \geq (p+\delta) \cdot t$ (with $0 < \delta < 1-p$) is
fulfilled, i.e., when the ratio between $0$ and $1$ is over the threshold
$p+\delta$.
Quite intuitively the sum will converge to $Y_t = p\cdot t$ (expected
value) but the probability of reaching $Y_t = (p+\delta)\cdot t$ is
non-zero, and will be reached in an infinite run. The question is
when will it be reached?
For this there are two interesting values I'd like to calculate: the
probability of each step $t \rightarrow t+1$ being the last, i.e.,
$Pr[Y_{t+1} \geq (p+\delta) \cdot (t+1)|Y_t < (p+\delta) \cdot t]$ and
the expected time until the sequence is terminated, $\mathrm{E}[\text{steps}]$.
 A: The expected time until the sequence is terminates is infinite.  In fact, not only is the expected value of $\text{min}\{t : Y_t>(p+\delta)t\}$ infinite for any $\delta>0$, but even the expected value of $\text{min}\{t : Y_t > pt + \delta\}$ is infinite.  
This can be proven using the optional sampling theorem (also known as optional stopping theorem).  Define $\tau = \text{min}\{t : Y_t > pt + \delta\}$, and define $Z_t=Y_t-pt$.  Note that 


*

*$Z_t$ is a martingale, and

*By definition, $Z_\tau\geq\delta>0$ and therefore $E[Z_\tau]>0$.


If $E[\tau]<\infty$ then the optional sampling theorem would imply that $E[Z_\tau]=Z_0=0$, which is a contradiction. 
The situation for the random time you mention in the OP is even more severe.
Let $K_t$ denote the event $Y_t>(p+\delta)t$ and define $\sigma=\text{min}\{t : K_t\}$. It can be shown that $P[\sigma=\infty]>0$, meaning that this threshold might never be crossed.  
Briefly, again we can do this by contradiction.  First, assume $P[\sigma<\infty]=1$, and let $\sigma_2$ denote the second time that 
$K_t$ occurs.  
Note that, for any $t$, if $Y_t-Y_\sigma>(p+\delta)(t-\sigma)$, then, since $Y_\sigma>(p+\delta)\sigma$ by assumption, it must be the case that $Y_t>(p+\delta)t$.  So, by the independence of the $Y_t$, it follows that
$P[\sigma_2-\sigma_1<\infty]=1$ and thus $P[\sigma_2<\infty]=1$ also.  And the same goes for $\sigma_3$ and $\sigma_4$, and so on, which means that the event $K_t$ occurs infinitely often with probability $1$.
However, since each $Y_t$ is the sum of independent Bernoulli trials, then it can be shown (i.e. Chernoff bounds) that there exists a fixed constant $C$ (depending on the constants $p$ and $\delta$) such that 
$$P[K_t]=P[Y_t-pt>\delta t]<e^{-Ct}.$$
From this it follows that
$$\sum_{t=1}^\infty P[K_t]<\infty,$$
and thus by the Borel-Cantelli lemma, with probability $1$, $K_t$ only occurs finitely often.  Hence we have a contradiction.
As for calculating the probabilities that each step is the last (i.e. the probabilities $P[\sigma=t]$), I can't think of any easy way to do this without doing a simulation.  That doesn't mean that there isn't any way to do it, of course. 
If you are looking for more information related to this question, I would suggest looking into the continuous version of the process, which would be brownian motion with drift.  I imagine you can find somewhere a calculation of the probability that brownian motion with drift crosses this kind of a threshold.
To add: if you are interested in numerical values, they are pretty easy to compute, at least for moderate values of $T$.  Here is a simple R function that computes the probability of escape, as well as the conditional distribution conditional on survival up to time n:
simProb<-function(p,n,delta){
  pp<-c(p,1-p)
  thresh<- floor(1+(p+delta)*(1:n))
  x<-1
  res<-rep(0,n)
  rsum<-1
  for(i in 1:n){
    x<-convolve(x,pp,t="o")
    if(length(x)>thresh[i]){      
      res[i]<-sum(x[(thresh[i]+1):length(x)])*rsum
      rsum<-rsum-res[i]
      x<-x[1:(thresh[i])]
      x<-x/sum(x)
    }
  }
  list(p=res,x=x)
}

Here is what the survival probabilities are for some specific constants:
plot(1-cumsum(simProb(0.6,1000,0.01)$p),type="l",ylim=c(0,0.4))


You can also examine the survival probability as a function of $\delta$ like this:
delt<-0.001*(1:399)
surv<-sapply(delt,  function(delta)1-sum(simProb(0.6,1000,delta)$p))
plot(delt,surv,type='l')


You can see how discontinuous this is: the reason is that, since $Y_t$ and $t$ are integer-valued the exit condition $Y_t>(p+\delta)t$ is effectively discontinuous as a function of $t$.  For this reason, it is unlikely that there is a clean formula for the probability of survival up to time $t$.
A: One of the question's queries is the probability that "the next step will be the last". The OP states this probability as $Pr[Y_{t+1} \geq (p+\delta) \cdot (t+1)|Y_t < (p+\delta) \cdot t]$, but this is not a correct representation. And this is because we require that the process must stop the first time it reaches the relevant threshold. Left unstopped, the process may cross the threshold at time $t$,then fall below the threshold that holds for period $t+1$, and then cross the next threshold again, etc. Namely we may have $Y_t < (p+\delta) \cdot t$, but $Y_{t-1} > (p+\delta) \cdot (t-1)$,  in which case $Y_t$ (and of course $Y_{t+1}$) won't be observed since that process will have stopped at $Y_{t-1}$.
To compact notation, denote $h_t$ the threshold of step $t$. Then the appropriate statement of the required probability is 
$$ Pr[Y_{t+1} \geq h_{t+1}\mid Y_{t+1}\; \text{is observed}]\equiv P^s_{t+1} $$
So first we need to calculate the continuation probability of the process. 
Counting from $1$, the probability that $Y_2$ will be observed is equal to the probability that at the first step, the Bernoulli trial will give a zero value: $Pr[Y_{2} \;\text{is observed}] = Pr(Y_1 =0) =1-p$.
The probability that $Y_3$ is observed is equal to the probability that $Y_2$ does not reach or exceed $h_2$, given that $Y_2$ is observed.
$$P^c_3 \equiv Pr[Y_3 \;\text{is observed}] = Pr[Y_2 < h_2 \mid Y_2 \;\text{is observed}]$$
Applying Bayes' theorem, we have 
$$Pr[Y_2 < h_2 \mid Y_2 \;\text{is observed}] = Pr[Y_2 \;\text{is observed} \mid Y_2 < h_2 ]\frac {Pr[Y_2 < h_2]}{Pr[Y_2 \;\text{is observed}]}$$
$$=1\cdot\frac {Pr[Y_2 < h_2]}{Pr[Y_2 \;\text{is observed}]} \Rightarrow P^c_3 = Pr[Y_2 < h_2]\cdot \left(P^c_2\right)^{-1}$$
So the probability that the $t+1$ step will be the last is
$$P^s_{t+1} = Pr[Y_{t+1}\; \text{is observed}\mid Y_{t+1} \geq h_{t+1}] \frac {Pr[Y_{t+1} \geq h_{t+1}]}{Pr[Y_{t+1}\; \text{is observed}]}$$
$$= Pr[Y_{t+1} \geq h_{t+1}]\cdot \left(P^c_{t+1}\right)^{-1}$$
Denote for compactness the unconditional probability of not reaching the threshold at trial $t$, $P^b_t$. These are probabilities regardless of whether the process may have crossed the threshold at some step before $t$. Then, using the recursive formula for $P^c_{t+1}$ we obtain
$$P^s_{t+1} = (1-P^b_{t+1})\frac{P^b_{t-1}\cdot P^b_{t-3}\cdot...}{P^b_{t}\cdot P^b_{t-2}\cdot...} \qquad [1]$$
Equation $[1]$ contains unconditional probabilites of sums (of different cardinality) of i.i.d. Bernoulli variables. The probability generating function of a Bernoulli r.v. is $G(z) = 1-p +pz$. The pgf of the sum of i.i.d Bernoullis is just the product of the individual pgf's. So
$$G_{Y}(z;t) = (1-p+pz)^t$$
The probability of $Y_t$ having the integer value $0\le k\le t$ (its pmf, that is) is recovered through the $k$-th derivative of the pgf evaluated at zero
$$P(Y_t=k) = \frac {1}{k!}G^{(k)}_{Y}(0;t) = \left(\begin{matrix} t \\k  
\end{matrix}\right) (1-p)^{t-k}p^k$$
and so 
$$P(Y_t\le k) = \sum_{j=0}^k \left(\begin{matrix} t \\j  
\end{matrix}\right) (1-p)^{t-j}p^j$$
In our case the threshold $h_t$ is not, in general, an integer. In order to cover both the integer and non-integer case, we use the ceiling function of $h_t-1$,  $\lceil h_t-1 \rceil$, obtaining for example,
$$P^b_{t} = Pr[Y_{t} \le \lceil h_t-1 \rceil] =  \sum_{j=0}^{\lceil h_t-1 \rceil} \left(\begin{matrix} t \\j  
\end{matrix}\right) (1-p)^{t-j}p^j \qquad [2]$$
a formula through which we can obtain the various components of $[1]$, for each step, and be able to calculate the required probability $P^s_{t+1}$ for any $\{p,\delta, t\}$. 
