It seems that you're missing the main point that Type I error rate is also your criterion for cutoff. If your criterion for cutoff is not changing then alpha is not changing.
The $p$-value is the conditional probability of observing an effect as large or larger than the one you found if the null is true. If you select a cutoff $p$-value of 0.05 for deciding that the null is not true then that 0.05 probability that it was true turns into your Type I error.
As an aside, this highlights why you cannot take the same test and set a cutoff for $\beta$. $\beta$ can only exist if the null was not true whereas the test value calculated assumes it is.
Frank Harrell's point is excellent that it depends on your philosophy. Nevertheless, even under frequentist statistics you can choose a lower criterion in advance and thereby change the rate of Type I error.