The assumption of normality in the t-test is about the specific sample that you have, not a theoretical distribution. Just because they should be theoretically normal doesn't mean they are, as you suggest from your own examination of the distribution. Therefore, you've violated the t-test assumptions and the Wilcoxon is a better choice. This is pretty common with a small N. It's unclear from the way you've worded things what the N in each group is. Is it a paired test?
It sounds like you're saying you have a few t-tests to do. If that's the case then you should probably use the non parametric test for all of them. It doesn't sound like you want to really estimate a parameter, but just check significance. Therefore, you wouldn't be getting anything extra out of the parametric test and it's not doing what you want when you violate it's assumptions.
Furthermore, if you really do have a number of tests to do within the same experiment then what your'e calling significant might not be once alpha is adjusted for the multiple comparisons. But theory would drive what needs to be done there and you haven't given us any.
Further to Felix S's answer, he implies that you should examine the data to see if the parametric effect was driven by particular data points. There might be something meaningful you can say about that.