Suppose I have two groups of data. I want to know if their distributions and / or means are different or not. Is it appropriate to do a two-sample t-test? Or should I go for a 2 sample goodness of fit test like 2D-KS test?
The basic principle is to use each test for what it was designed to do. Here are specific examples:
Example 1: Use t test. If you're trying to see whether two normal samples have the same mean, then use a t test in preference to a K-S test. This is especially true if samples are small so that the K-S test has relatively poor power. In the example below, the t test rejects $H_0$ at the 5% level, and the K-S test does not.
set.seed(706) x1 = rnorm(15, 100, 15) x2 = rnorm(15, 120, 15) t.test(x1,x2)$p.val  0.04862413 ks.test(x1,x2) Two-sample Kolmogorov-Smirnov test data: x1 and x2 D = 0.4, p-value = 0.1844 alternative hypothesis: two-sided mean(x1); sd(x1)  101.7776  20.05416 mean(x2); sd(x2)  115.5439  16.2465
The K-S test compares sample ECDFs. The K-S test statistic $D$ is the maximum vertical distance between the two ECDFs. For significance in small samples, $D$ must be fairly large. [There is a shift in means, but the vertical distance betwee the the ECDFs is never great--relative to the jump sizes at each observation.]
plot(ecdf(x1), col="blue", main="ECDFs of Two Samples") lines(ecdf(x2), col="orange")
Example 2: Use K-S test. If you're trying to distinguish between shapes of distributions that have similar means and variances, then use the K-S test in preference to the t test. In the example below, the K-S test rejects $H_0$ at the 5% level, and the t test does not.
set.seed(2020) y1 = rnorm(85, 1, 1) y2 = rexp(85,1) summary(y1); sd(y1) Min. 1st Qu. Median Mean 3rd Qu. Max. -2.0388 0.4949 1.1188 1.1171 1.7206 4.2016  1.094069 summary(y2); sd(y2) Min. 1st Qu. Median Mean 3rd Qu. Max. 0.008152 0.282684 0.764139 1.049091 1.570979 4.116282  0.982902 t.test(y1,y2)$p.val  0.6702003 ks.test(y1,y2) Two-sample Kolmogorov-Smirnov test data: y1 and y2 D = 0.21176, p-value = 0.04395 alternative hypothesis: two-sided plot(ecdf(y1), col="blue", main="ECDFs of Two Samples") lines(ecdf(y2), col="orange")