No this is not true.
One-sided case
Therefore, if this tolerance bound is higher than $y_0$, one can say
that the exceedance probability $\Pr(Y > y_0)$ is
$\alpha$-significantly higher than $p$.
Let's check this claim with the help of simulations. We simulate $Y_i \sim_{\text{iid}} \mathcal{N}(0,1)$, we take $y_0=1$, and we take the lower tolerance bound with coverage $p=\Pr(Y_i>y_0)$. Then the probability that the lower tolerance bound is higher than $y_0$ should be $\alpha$.
y0 <- 1
p <- 1-pnorm(y0)
nsims <- 200000
n <- 10
k <- tolerance::K.factor(n, alpha=0.05, P=p, side=1)
test <- logical(nsims)
for(i in 1:nsims){
Y <- rnorm(n)
bound <- mean(Y) - k*sd(Y)
test[i] <- bound > y0
}
This is confirmed by the simulations:
> mean(test)
[1] 0.049805
Two-sided case
Now take two real numbers $y_1 < y_2$. If the two-sided
$(1-\alpha,p)$-tolerance interval falls inside the interval
$[y_1,y_2]$, does it make sense to say that the probability
$\Pr\bigl(Y \in [y_1, y_2]\bigr)$ is significantly higher than $p$ at the
$\alpha$ level?
Let's check with simulations that this is not true. As for the one-sided case, if we take $p = \Pr\bigl(Y_i \in [y_1, y_2]\bigr)$, then the probability that the two-sided $(1-\alpha,p)$-tolerance interval falls inside the interval $[y_1,y_2]$ should be $\alpha$ if the claim were true.
y1 <- -1; y2 <- 2
p <- pnorm(y2)-pnorm(y1)
nsims <- 200000
n <- 10
k <- tolerance::K.factor(n, alpha=0.05, P=p, side=2, method="EXACT")
test <- logical(nsims)
for(i in 1:nsims){
Y <- rnorm(n)
bounds <- mean(Y) + k*c(-1,1)*sd(Y)
test[i] <- bounds[1] > y1 && bounds[2] < y2
}
This is not confirmed:
> mean(test)
[1] 0.01193