3
$\begingroup$

SEM involes testing the significance of the relationship of dozens if not scores of observed variables with latent variables. After this, the relationship between several latent variables are tested as path coefficients. Evere time a statistical test is conducted the probility of a false positive increases. Here are my questions

  1. Is it possible that so many statistical test being ran on the same data is an example of p-hacking?
  2. Does any SEM software correct for this ie bonferroni correction?
  3. Is this problem not a concern in the social sciences were SEM is used frequently?
Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

  1. Yes. But it's the opposite of p-hacking in some ways. Researchers seek non-statistically significant results, because they want their models to fit.

  2. Not that I know of.

  3. Yes. Researchers think that they can't publish a model until it fits. They try to tweak their models until they do fit.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I don't think OP is referring to goodness of fit tests but rather the to the Wald tests of each coefficient estimated in the model. The chi-square goodness of fit test, for example, is not subject to p-hacking if only one model is run, whereas the probability of falsely rejecting the null hypothesis for any single path can be high when many coefficients are being tested. $\endgroup$
    – Noah
    Commented Sep 13, 2018 at 2:05
  • $\begingroup$ I agree, but no one only runs one model. The main significance test people worry about is the chi-square test. $\endgroup$
    – Jeremy Miles
    Commented Sep 13, 2018 at 3:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.