Meaning of significance level and critical values for kolmogorov smirnov goodness of fit test I'm new to using goodness of fit tests and am encountering what I guess is a beginners misunderstanding of how the tables for critical values and different significance levels have to be read. I have searched for similar questions but did not find an answer that would help me.
In the tables, such as this one, the critical values at a given sample size increase with (1 - alpha). It was my understanding that the higher 1 - alpha (or the lower alpha), the more likely my hypothesis is to be true. If this is so, I don't understand why the test statistics have to be smaller than the critical values in order to not reject the hypothesis. Something seems to be upside down here. Could you please point out my mistake?
Thanks in advance!
 A: Hypothesis testing can be tricky to understand.   You wrote "the more likely my hypothesis is to be true", but this isn't how it works.
You assume a model (called the "null model"), and then see if there is evidence to reject it.  That is, you ask "how surprised would I be to see the observed data, if the hypothesis really were true".  
The "how surprised" you are is measured by the "level" $\alpha$, where a smaller $\alpha$ means more surprised.  The critical value (at which you reject the hypothesised model, due to being so shocked at what you observe) is determined by $\alpha$.
To reject at small level $\alpha$, you need to observe a more shocking test statistic, which is what the higher critical value says.
A: 
It was my understanding that the higher 1 - alpha (or the lower alpha), the more likely my hypothesis is to be true. If this is so,

It isn't so.
The critical value at level $\alpha$ is the value of the test statistic beyond* which only a proportion $\alpha$ of test statistics will lie, if the null hypothesis is true. 
* (in whichever direction or directions are consistent with the alternative)
$D$ measures the distance the two distribution functions (the hypothesized and the observed) are apart. Larger D values are more consistent with the alternative.
So when you increase the critical $D$ value, $\alpha$ must be smaller, since the proportion beyond it must be smaller.
If D is smaller than the critical value it indicates that the two distributions aren't so different.
