# Simulating multiple comparisons example

I referred to this question, but found no solution to my problem.

I want to simulate a simple case of multiple comparison problem, where family-wise alpha error increases.

# Question 1

First example includes only a single test:

library (tidyverse)

# 100 datasets with 100 participants, means and sd
set.seed(1)
A <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))
B <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))
C <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))

test1 <- map2_dbl(A, B, ~t.test(.x, .y)$p.value) sum(test1<=0.05 )/length(test1) # 0.044  However, when i try to simulate a case where I compare A-B, A-C and B-C with t-tests, I don't get the expected amount of significant p-values $$\alpha_F=1-(1-0.05)^c$$, where c is the number of comparisons (here - 3). Family-wise alpha should be $$\alpha_F=0.1426$$. Here's the code: triple_t <- function(x,y,z) { t1 <- t.test(x,y) t2 <- t.test(x,z) t3 <- t.test(y,z) return(c(t1$$p.value, t2$$p.value, t3$p.value))
}

test2 <- pmap(list(A, B, C), triple_t) %>% flatten_dbl()
sum(test2<=0.05 )/length(test2) # 0.047


What am I missing here?

# Question 2

If we have a situation with 6 different datasets, all drawn from the same population

set.seed(1)
A <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))
B <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))
C <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))
D <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))
E <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))
F <- rerun(.n = 1000, rnorm(n = 100, mean = 0, sd = 1))


would you expect the same family-wise error probability to increase if you conduct comparisons only on independent samples (A-B, C-D, E-F)?

triple_talt <- function(a,b,c,d,e,f,g) {
t1 <- t.test(a,b)
t2 <- t.test(c,d)
t3 <- t.test(e,f)