Potential problems when doing hypothesis testing on large number of variables I encounter this question in an interview.
We were discussing AB testing on one categorical variable and this was the follow up question: what if now instead of one variable we want to test 100 variables.  What are the potential problems that could occur in hypothesis testing if we have large number of variables and how do we address the issue?
 A: I will give some thoughts on all three issues I've raised.
100 covariates
An issue here is that you will need a lot of data to get reliable estimates of the parameters in your regression. Otherwise, your model can badly overfit, meaning that you're basically guessing when you do inference on the group parameter. A solution is to collect more data or to be more decisive about what covariates matter the most. Don't chuck something at the regression just because you have it!
100-variate distributions
(The intuition behind this comes from Hotelling's $T^2$ test.)
If your multivariate distributions have significant differences in just one marginal distribution, then that can get drowned out by the small differences. Getting more technical about that, and using Hotelling's T^2 test as an example, you increase the degrees of freedom without increasing the value of the test statistic by enough to justify such an increase. (However, if two distributions are the same, then their marginal distributions must be the same!) A solution is, again, to be more decisive about what variables matter the most. Don't chuck something at the problem just because you have it! Another solution is to do marginal tests to see which variables are non-contributors. However, then you're doing many comparisons, which leads to...
100 univariate tests
I think this is what you meant. There are many ways of addressing this. The simple one is a Bonferroni correction, where you test at the $\alpha/k$-level if you want to do $k$-many (such as $100$) tests at the $\alpha$-level (such as $0.05$). A number of improvements upon Bonferroni exist. One is Bonferroni-Holm, which dominates Bonferroni in the sense that it never performs worse but sometimes performs better. The only reason to use Bonferroni is convenience of the calculation. A more sophisticated method is the Benjamini-Hochberg method for controlling the false discovery rate.
