You are right to worry, the equality
$$\lim_{n\to\infty} P(X_n-\sqrt{n}c<t-\sqrt{n}c) = \lim_{n\to\infty} \Phi(t-\sqrt{n}c)$$
is not correct, as the term $t-\sqrt n c$ depends on $n$ (see the definition of convergence in distribution here).
It's probably overkill, but here is an argument using Skorokhod's representation theorem : this theorem tells us that there exists a probability space $(\Omega,\mathcal F,\mathbb Q)$, and random variables $\tilde Z\sim\mathcal N(0,1), \tilde X_1, \tilde X_2,\ldots$ defined on it such that for all $n\ge 1$, $X_n$ and $\tilde X_n$ have the same distribution and
$$\tilde X_n -\sqrt n c\to \tilde Z\quad \ \mathbb Q\text{-a.s.} $$
Now because $X_n$ and $\tilde X_n$ have the same law for all $n$, we can say for all $t>0$ :
\begin{align}
P(X_n\le t) &= \mathbb Q(\tilde X_n \le t)\\
&= \mathbb Q(\tilde X_n - \sqrt n c \le t -\sqrt n c)\\
&= \mathbb Q(\tilde X_n - \sqrt n c -\tilde Z + \tilde Z \le t -\sqrt n c)\\
&\le \mathbb Q(\tilde X_n - \sqrt n c -\tilde Z \le (t -\sqrt n c)/2)\tag1\\
&+ \mathbb Q(\tilde Z \le (t -\sqrt n c)/2)\tag 2\\
\end{align}
Where we used the fact that $\{X+Y\le k\}\subseteq \{X\le k/2\}\cup\{Y\le k/2\}$ together with the union bound for the last inequality.
For term $(1)$, we have (provided that $c>0$) that for all $n\ge (t+2)/c$, $(t -\sqrt n c)/2 \le -1 $, hence for all large enough $n$
\begin{align}
\mathbb Q(\tilde X_n - \sqrt n c -\tilde Z \le (t -\sqrt n c)/2) &\le \mathbb Q(\tilde X_n - \sqrt n c -\tilde Z \le -1)\\
&=\mathbb Q(\tilde Z - \tilde X_n + \sqrt n c \ge 1)\\
&\le \mathbb Q(|\tilde Z - (\tilde X_n - \sqrt n c) |\ge 1)
\end{align}
and this last term goes to zero as $n\to\infty$ since a.s. convergence implies convergence in probability.
For term $(2)$, simply recall that we have
$$ \mathbb Q(\tilde Z \le (t -\sqrt n c)/2) = \Phi((t -\sqrt n c)/2)$$
which also goes to zero as $n\to\infty$, so unless I made a mistake, we are done.
The result does not hold for $c\le 0$, as a counterexample given by @whuber shows : for all $n\ge1$, let $X_n = Z - \kappa\sqrt n$ where $Z\sim\mathcal N(0,1)$ and $\kappa\ge 0$, so that the sequence $X_n - (-\kappa\sqrt n) = Z$ obviously converges in distribution to a standard normal.
In this case however we have $$P(X_n>t) = P(Z>t + \kappa\sqrt n) = 1 - \Phi(t + \kappa\sqrt n) \not\to 1. $$
