I want to compare two independent samples (from A/B test). T-test is most sensitive for normally distributed data (and will give smaller p-value than U-test if there is a real shift in mean value). But T-test performs poorly when there are large outliers: it would give high p-values. And here U-test is much better. So in practice one should look at data, decide how normal are the values and whether the outliers would be a problem. And then choose which test to use.
But if T-test would give smaller p-values for normally distributed data, while U-test for other. can I make a decision about how normal are the data by comparision p-values of T-test and U-test?
And finally I just calculate p-values from both tests and pick the smallest without looking at data. If they are normally distributed, than T-test would give the smaller p-value (which is good), in other case the U-test will give the smaller p-value (which is again good). So sounds like a universal approach. Is it a good idea? Why haven't I head that solution as a best practice for those who wants best results without digging into data?
I've tested this idea by generating random data with different distributions and for all examples I've made the combined test was correct. So if you state it is wrong approach please help with counterexample.