For the KS test the p-value is itself distributed uniformly in [0,1] if the H0 is true (which it is if you test whether it your sample is from $U(0,1)$ and the random number generation works okay). It therefore must "vary wildly" between 0 and 1, in fact its standard deviation is $1/\sqrt{12}$ which is roughly 0.3.

You can check this by looking whether the percentages of p values smaller or equal to some $p_0$ over your independent consecutive runs is close to said $p_0$.

See also Why are p-values uniformly distributed under the null hypothesis?

For the KS test the p-value is itself distributed uniformly in [0,1] if the H0 is true (which it is if you test whether it your sample is from $U(0,1)$ and the random number generation works okay). It therefore must "vary wildly" between 0 and 1, in fact its standard deviation is $1/\sqrt{12}$ which is roughly 0.3.

You can check this by looking whether the percentages of p values smaller or equal to some $p_0$ over your independent consecutive runs is close to said $p_0$.

See also Why are p-values uniformly distributed under the null hypothesis?