You should indeed not interpret the p-value as a probability that the null hypothesis is true.
However, a higher p-value does relate to stronger support for the null hypothesis.
Considering p-values as a random variable
You could consider p-values as a transformation of your statistic. See for instance the secondary x-axis in the graph below in which the t-distribution is plotted with $\nu=99$.

Here you see that a larger p-value corresponds to a smaller t-statistic (and also, for a two-sided test, there are two t-statistic associated with one p-value).
Distribution of p-values $P(\text{p-value}|\mu_1-\mu_2)$
When we plot the distribution density of the p-values, parameterized by $\mu_1-\mu_2$, you see that higher p-values are less likely for $\mu_1-\mu_2 \neq 0$.

# compute CDF for a given observed p-value and parameter ncp=mu_1-mu_2
qp <- function(p,ncp) {
from_p_to_t <- qt(1-p/2,99) # transform from p-value to t-statistic
1-pt(from_p_to_t,99,ncp=ncp) + pt(-from_p_to_t,99,ncp=ncp) # compute CDF for t-statistic (two-sided)
}
qp <- Vectorize(qp)
# plotting density function
p <- seq(0,1,0.001)
plot(-1,-1,
xlim=c(0,1), ylim=c(0,9),
xlab = "p-value", ylab = "probability density")
# use difference between CDF to plot PDF
lines(p[-1]-0.001/2,(qp(p,0)[-1]-qp(p,0)[-1001])/0.001,type="l")
lines(p[-1]-0.001/2,(qp(p,1)[-1]-qp(p,1)[-1001])/0.001,type="l", lty=2)
lines(p[-1]-0.001/2,(qp(p,2)[-1]-qp(p,2)[-1001])/0.001,type="l", lty=3)
The bayes factor, the ratio of the likelihood for different hypotheses is larger for larger p-values. And you could consider higher p-values as stronger support. Depending on the alternative hypothesis this strong support is reached at different p-values. The more extreme the alternative hypothesis, or the larger the sample of the test, the smaller the p-value needs to be in order to be strong support.

Illustration
See below an example with simulations for two different situations. You sample $X \sim N(\mu_1,2)$ and $X \sim N(\mu_2,2)$ Let in one case
- $\mu_i \sim N(i,1)$ such that $\mu_2-\mu_1 \sim N(1,\sqrt{2})$
the other case
- $\mu_i \sim N(0,1)$ such that $\mu_2-\mu_1 \sim
N(0,\sqrt{2})$.

In the first case you can see that the probability for $\mu_1-\mu_2$ is most likely to be around 1, also for higher p-values. This is because the marginal probability $\mu_1-\mu_2 \sim N(1,\sqrt{2})$ is already close to 1 to start with. So a high p-value will be support for the hypothesis $\mu_1-\mu_2$ but is is not strong enough.
In the second case you can see that $\mu_1-\mu_2$ is indeed most likely to be around zero when the p-value is large. So, you could consider it as some sort of support for the null hypothesis.
So in any of the cases a high p-value is support for the null hypothesis. But, it should not be considered as the probability that the hypothesis is true. This probability needs to be considered case by case. You can evaluate it when you know the joint distribution of the mean and the p-value (that is, you know something like a prior probability for the distribution of the mean).
Sidenote: When you use the p-value in this way, to indicate support for the null hypothesis, then you are actually not using this value in the way that is was intended for. Then you may better just report the t-statistic and present something like a plot of a likelihood function (or bayes factor).