On an implication of the memoryless property of the exponential random variable I know that if we take $X \sim Exp(k)$ then we have this property:
$$P(X \ge s + t | X \ge s) = P(X \ge t)$$ 
But why does this imply that $X | X > x$ has the same distribution of $X$ only shifted to the right by $x$?
 A: For any real number $x$ and $s \geq 0$,
$$\begin{align}
 P\{X > x\mid X > s\} 
&= \frac{P\left(\{X > x\}\cap \{X > s\}\right)}{P\{X > s\}}\\
&= \frac{P\{X > \max\{x, s\}\}}{P\{X > s\}}.\tag{1}
\end{align}$$
But, 
$\displaystyle\max\{x,s\} = \begin{cases}x, & x \geq s,\\s, &x < s,\end{cases}$
while $P\{X > t\} = \begin{cases}e^{-\lambda t}, &t \geq 0,\\1, & t < 0, \end{cases}$ and so $(1)$ becomes
$$P\{X > x \mid X > s\} = 
\begin{cases}e^{-\lambda (x-s)}, & x \geq s,\\1, &x < s,\end{cases}$$
leading to
$$F_{X\mid X > s}(x) =  
\begin{cases}1 - e^{-\lambda (x-s)}, & x \geq s,\\0, &x < s.\end{cases}\tag{2}$$
Comparing this to the unconditional CDF 
$F_X(x) = \begin{cases}1 - e^{-\lambda x}, & x \geq 0,\\0, &x < 0,\end{cases}$
we see that we have that

$$F_{X\mid X > s}(x) = F_X(x-s),\tag{3}$$
  that is, the conditional CDF of $X$
  conditioned on the event $\{X > s\}$, is the same as the
  unconditional CDF of $X$ displaced to the right by $s$.

As Xi'an has already pointed out to you, you cannot (sensibly)
write $X = X+s$ or that the
CDF of $X-s$ is the same as the CDF of $X$ etc. Nor does it make
sense to think of $X\mid X>s$ as a random variable; it is not. The only random
variable here is $X$. What you can (and should!)
say is what is shown in the highlighted text above.
