For example, I want to do test on each variable independently. Some variables are normally distributed, some are log-normally distributed, some just can't be transform to normal distribution (not even by Box-Cox, etc.). Because for each condition, T-test, T-test after log transformation, non-parametric T-test gain highest power than the other two methods. Can I simply use different method for different situation because of high power?
I guess the answer is No because of increment of Type I error. But I just cannot understand how does this increase Type I error. Isn't it true that we should use right test (the test that makes right assumption)?
To make question more clear. Suppose I have a metabolomics data set, which contains two group, group health (size:15) and group ill (size:15). Columns are 1000 metabolites. So data set is 30*1000. I am trying to figure out which metabolites may be influenced by the treatment. First I do univariate analysis, t-test on each metabolite (More suggestion on statistical methods for omics data analysis is more than welcomed). We don't know about the distribution of each metabolite. So we can only make assumption of normality based on the data we have. So what I try to do here is for each metabolite, if the metabolite is normally distributed , then use t-test; if the metabolites can be transform to normal distribution, after transformation use t-test. Otherwise use non-parametric method for comparing two group means (non-parametric t-test is not a good word).
My question is that, can I do so? Thank you!
What I am doing is not trying to use the statistical test which gives the smallest p-value for each variable. I believe this is not correct.