What is the distribution and standard deviation of the number of Type I errors in hypothesis testing? My question is from the book "An introduction to statistical learning" (2nd edition), chapter 13, exercise 2.
Suppose that we test m hypotheses, and control the Type I error for each hypothesis at level α. Assume that all m p-values are independent, and that all null hypotheses are true. What is the distribution and standard deviation of the number of Type I errors that we will make?
Here is my approach:
Step 1: Simulate an (n, m) data set where the true mean is zero.
n = 20
m = 100
X = matrix ( rnorm (n * m) , ncol = m)

Step 2: Implement t-test on X using α = 0.05 and record the number of Type I errors.
Step 3: Repeat step 1 and 2 about 1000 times, which means I have 1000 numbers of Type I errors.
I plot a histogram of the 1000 numbers of Type I errors, it looks like a normal distribution or a t distribution. I have no idea if my approach is correct and if it is, how can I calculate the standard deviation?
If my approach is incorrect, how can I solve the problem?
 A: Type one error is that we reject $H_0: \text{the mean of population} = 0$ while $H_0$ is true. We genecrate the matrix $X$ and apply t test to $X$ by column. Then we get the number of rejecting $H_0$ for one specific $X$. We repeat the step 1000 times and get 1000 numbers.
set.seed(1234)
typeOneErrNum <- c()
for(i in 1:1000){
  n <- 20
  m <- 100
  X <- matrix(rnorm(n * m), ncol = m)
  p.value <- apply(X, 2, FUN = function(x){t.test(x)$p.value})
  typeOneErrNum <- c(typeOneErrNum, sum(p.value < 0.05))
}
hist(typeOneErrNum)


A: Writing out the hint: Each of your $m$ independent tests is a Bernoulli experiment, since the nulls are true your probability of rejection is equal to the size $\alpha$. Again, since all nulls are true, all rejects is a Type I error. By definition, the Binomial distribution is the distribution of total number or successes (in this case, a "success" is a reject) in $m$ independent Bernoulli experiments, all with the same probability of success, here $\alpha$.
The mean and variance of a binomial distribution have known, simple formulas, so you can easily find the standard deviation.
