Ok since whuber [commented](https://stats.stackexchange.com/questions/651379/how-is-summation-by-parts-technique-used-in-this-derivation#comment1223427_651379), I decided to look back into it and figured it out.

I think the following expression from wikipedia makes it easier to understand:
$$
\sum_{k=0}^n f_k g_k = f_0 \sum_{k=0}^n g_k + \sum_{j=0}^{n-1} (f_{j+1} - f_j) \sum_{k=j+1}^n g_k.
$$

Also as whuber hinted:

$$
f_k =  \sum_{i'=0}^{k-1} 1 . 
$$

So the first part of the definition is 
$$ 
\sum_{k=0}^n f_k g_k  = \sum_{k=0}^{\infty}\left ( P(X=k)  \times \sum_{i'=0}^{k-1} 1 \right)
$$

So the equivalent is, 

$$
f_0 \sum_{k=0}^n g_k + \sum_{j=0}^{n-1} (f_{j+1} - f_j) \sum_{k=j+1}^n g_k =\sum_{i'=0}^{0} 1 \times \sum_{k=0}^{\infty} P(X=k) + \sum_{j=0}^{\infty} \left(\sum_{i'=0}^{j+1} 1 - \sum_{i''=0}^{j} 1 \right) \sum_{k=j+1}^n P(X=k)
$$

$$
= 0 + \sum_{j=0}^{\infty} 1 \times \sum_{k=j+1}^{\infty} P(X>j) = \sum_{j=0}^{\infty} P(X>j) = \sum_{j=0}^{\infty} 1 - P(X<j). 
$$