A distribution can be non-normal and yet have identical skewness and kurtosis to a normal distribution
The fourth image in this answer is one example - a symmetric "double gamma" distribution with shape parameter around 2.303. Even symmetry is not guaranteed with skewness 0.
Even when population skewness and kurtosis are quite different from those for the normal, at small sample sizes it can be very hard to judge whether that's the case. In larger samples, you might be able to distinguish those cases ... but they may not matter so much then.
A few more specific points
1: skewness and kurtosis alone are not necessarily a good way to judge whether the sample is consistent with normality
2: using formal tests to assess normality for the purpose of choosing which test to apply answers the wrong question. Your data aren't going to be from a normal distribution. The relevant question might be something like "how non-normal might they be and how much impact will that have on significance level and power?"
In large samples, you'll reject normality when it doesn't matter and in small samples you'll fail to reject when it may matter. The test rarely tells you anything useful.
3: several papers advise against trying to choose which test to do based on goodness of fit tests, because it affects the behavior of the subsequently chosen test. If you don't have some reason to think the normal-theory procedures would be a reasonable choice, simply don't assume normality. There are more choices than t-tests and Wilcoxon-Mann-Whitney.