So, lets compare two normal distributions

Do this x times: 

runs <- 100000
a.samples <- rnorm(runs, mean = 5) 
b.samples <- rbeta(runs, mean = 0) 
mc.p.value <- sum(a.samples > b.samples)/runs

The mc.p.values falling below our alpha (0.05) divided by x would then give the type1 error rate. Our H0 is a.samples >= b.samples. (Inspired by https://www.countbayesie.com/blog/2015/3/3/6-amazing-trick-with-monte-carlo-simulations)

But, I thought a montecarlo simulation had to follow the following steps:

Algorithm:

1. Set up some distribution for the data, f() or f(θ), and some H0
2. Repeat the following two steps many times: (a) Simulate a data set according to H0 (b) Calculate T(x) using the simulated data
3. Add T(X) evaluated from the sample data
4. Order all of the T(x)s
5. p-value is the proportion of the T(x)s as extreme or more extreme than the one from the sample data

Therefore the first code snippet isn't a bona fide monte carlo simulation?

So, lets compare two normal distributions

Do this x times: 

runs <- 100000
a.samples <- rnorm(runs, mean = 5) 
b.samples <- rbeta(runs, mean = 0) 
mc.p.value <- sum(a.samples > b.samples)/runs

The mc.p.values falling below our alpha (0.05) divided by x would then give the type1 error rate. Our H0 is a.samples >= b.samples. (Inspired by https://www.countbayesie.com/blog/2015/3/3/6-amazing-trick-with-monte-carlo-simulations)

But, I thought a montecarlo simulation had to follow the following steps:

Algorithm:

1. Set up some distribution for the data, f() or f(θ), and some H0
2. Repeat the following two steps many times: (a) Simulate a data set according to H0 (b) Calculate T(x) using the simulated data
3. Add T(X) evaluated from the sample data
4. Order all of the T(x)s
5. p-value is the proportion of the T(x)s as extreme or more extreme than the one from the sample data

Therefore the first code snippet isn't a bona fide monte carlo simulation? and is the p-value valid, because, if you go to graph it, you don't get the expected 5% type1 error rate that one might expect for a statistical test.