When we collect genomics/metabolomics/proteomics data (many variables) in a between-by-within 2-way experimental design, we are often interested in which variables change differently between groups (interaction) and which variables change equally among groups (main effect).
A follow up question after the analysis is whether the overall effect is mostly different between groups or the same among groups i.e. how many significant interactions vs main effects.
However, main effects comparisons are more powerful than interaction tests, so equivalency across groups is more likely to be detected than interactions.
Can anyone recommend an approach to put these tests on the same playing field? Simply use a more stringent p-value (or false discovery rate) for the main effects? How would we choose the different cut-offs?
P.S. I would gladly change the unappealing title of "significance scale" if someone has a better term.