Following up on the very astute observation by Haffi112 and the comment by @whuber, consider a standard problem of mutually exclusive events $A$ and $B$ that can occur on a trial of a simple experiment. Mutually independent trials of the simple experiment are conducted until one of the two events occurs. What is the probability that event $A$ occurs before event $B$? The standard analysis says that because of independence, we can ignore all previous trials and look only at the trial on which we know that one of $A$ and $B$ occurred, that is, $A \cup B$ occurred. The conditional probability that $A$ occurred on this trial is exactly the probability of "$A ~\text{before}~ B$"
that we seek.
$$P\{A ~\text{occurs before}~ B\} = P(A \mid (A\cup B))
= \frac{P(A \cap (A \cup B))}{P(A \cup B)} = \frac{P(A)}{P(A) + P(B)}$$
What if the trials of the simple experiment are not independent?
The probabilities of $A$ and $B$ occurring on the $i$-th trial given that neither $A$ nor $B$ has occurred
on the first, second, $\ldots$, $(i-1)$-th trials do depend on what
happened on the previous trials, but denoting these probabilities
by $P_i(A)$ and $P_i(B)$ respectively, we have, as before, that
the conditional probability that the $i$-th trial concludes
the compound experiment via occurrence of $A$ is
$P_i(A)/(P_i(A) + P_i(B))$. Note that we are conditioning on what
happened on the previous trials.
In the problem being analyzed,
$$P_i(A) = \frac{k}{D}, ~ P_i(B) = \frac{(k + 1) + (k+2) + \cdots + n}{D}
= \frac{\binom{n+1}{2} - \binom{k+1}{2}}{D}
$$
where $D$ is the "weight" of the balls left in the urn after the $i-1$
trials. The exact value of $D$ depends on which of the balls numbered
$1, 2, \ldots, (k-1)$ was drawn on the previous $i-1$ trials, but the
ratio $P_i(A)/P_i(B)$ is the same regardless of what happened on
the previous trials (as Haffi112 observed), and thus the probability
that the compound experiment concludes on the $i$-th trial of the
simple experiment is the same no matter which of the balls numbered
$1, 2, \ldots, (k-1)$ was drawn on the previous $i-1$ trials.
The compound experiment must conclude no later than the $k$-th
trial of the simple experiment. The
law of total probability says that
$$
P\{A ~\text{occurs before}~ B\}
= \sum_{i=1}^{k} \frac{P_i(A)}{P_i(A) + P_i(B)}\times
P\{\text{experiment concludes on}~i\text{-th trial}\},
$$
but since
$$
\frac{P_i(A)}{P_i(A) + P_i(B)} = \frac{k}{\binom{n+1}{2} - \binom{k+1}{2}}
~ \text{for}~ i = 1, 2, \ldots k,
$$
we have that
$$
P\{A ~\text{occurs before}~ B\} = \frac{k}{\binom{n+1}{2} - \binom{k+1}{2}}
$$
Note that the result is slightly different from the one conjectured by Haffi112.
