I just came across an article (Jobst, Auerswald, & Moshagen, 2021) covering this topic, so I thought I would share it. The article is a simulation study, and they found that latent trait non-normality typically had a negligible effect on parameter estimates. However, it had a more significant impact on fit. In particular, the effect of latent trait non-normality on the model fit when asymptotically distribution-free (ADF) estimation was non-negligible. The effect of latent trait non-normality on model fit when Maximum Likelihood (ML) was used was negligible. The fact ML was more robust to non-normality is not particularly surprising, in light of recent research (e.g., Shi & Maydeu-Olivares, 2020; Xia & Yang, 2019) concerning the impact of estimation method on confirmatory factor analysis (CFA) and structural equation modeling (SEM) model fit.
Jobst, L. J., Auerswald, M., & Moshagen, M. (2021). The Effect of Latent and Error Non-Normality on Measures of Fit in Structural Equation Modeling. Educational and Psychological Measurement, 00131644211046201.
Shi, D., & Maydeu-Olivares, A. (2020). The effect of estimation methods on SEM fit indices. Educational and Psychological Measurement, 80(3), 421-445.
Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior research methods, 51(1), 409-428.