comparing normal distributions using a two sample kolmogorov-smirnov test I've been redirected here from another forum... 
I have used a two sample Kolmogorov-Smirnov test to compare the distributions of two sets of data. Basically I am comparing the error distributions between two measurements when an intervention is made, to determine whether the intervention (a changing measurement parameter) significantly changes the error distribution of the measurement.
I know that the K-S test is a non parametric test, however the distributions of data I'm comparing has turned out to be normally distributed... I know there is probably a number of tests that could be used to compare normally distributed data, but is there a reason not to use the K-S test? Are there any disadvantages (with regard to type1 and 2 errors perhaps)? Is it ok to use it?
I've sort of gone down this route with my data analysis, but the question has come up: why use a non parametric test to compare parametric data? Hopefully K-S is unconventional rather than completely wrong.
 A: There is nothing wrong with using the K-S test. The reason you might want to use other tests such as ones tailored from a Normal distribution is that these tests may give you more power. But if you are not certain of normality,  I think K-S test will do just fine. I personally like the K-S test because it has fewer assumptions compared to other tests. Back in the day when we didn't have the computing power that we do now, parametric test were easier computationally and valid if they fit the assumptions. But now with the computing power we have, there isn't to much of a reason not to do nonparametric tests
The one thing you will want to make sure is that the you have a large enough of sample size for both to ensure empirical CDF is close to actual CDF.
A: I think K-S test has an important limitation which requires the underlying distribution must be fully defined. That is why if mean and variance parameters are estimated from the actual data, K-S test are no longer valid and its critical values are difficult to get. You can have a look at some papers writing about this problem: Koul and Swordson (2011), Bai(2003)...
