Why don't we control for multiple comparisons in factor analysis?

How can we trust the significance tests for any of our factor loadings, or even the correlations matrix, when so many tests are run in factor analysis?

Especially for IRT models where we may be analyzing multiple items in a scale.

As I understand you can't trust $p$ values in significance tests of the factor loadings, though researchers mostly ignore this in practice. Whenever we have many hypotheses being tested simultaneously, the inflation of Type I error rate may be an issue. This question Multiple hypothesis testing in SEM deals with a similar issue in the context of structural equation modeling. In the article linked in the answer to that question, it is argued that confirmatory factor analyses should use a procedure for reducing Type I errors and that using false discovery rate procedures offers a good balance between Type I and Type II error rates.