2
$\begingroup$

I am currently trying to fit a structural equation model in R with the Lavaan package. I have this model that fits my data pretty good. This model is what I consider the full model, it has all paths that theoretically make sense. The CFI, RMSEA, SRMR are all within an acceptable range and the X2 is not significant so it seems that the model fits my data well.

There are however, some paths that are not significant. If i leave these non-significant paths out my model fits the data even better.

My problem is now that I am a little bit confused about model selection. Most things that I read about this are a bit vague. Some say that you should modify the full model as less as possible. As my full model is a good fit I could keep it as it is. Others say that I should remove the non-significant paths and compare the models with a likelihood ratio test, AIC, AICc or BIC scores. If I remove the non-significant paths from my model the estimates don't change very drastically but the AICc ends up 15 points lower which would mean that the reduced model is better.

If I would use the AICc scores to select my model shouldn't I than compare all possible combinations of models and select the best or even calculate an averaged model?

The question is, how should I select my final model (unchanged full model, AICc,...)?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

(I could not comment yet so I post as an answer...)

I read many paper that used SEM, here is what I have observed from the papers:

  1. Parsimony is better. If you are using SEM as EFA, usually people remove non-sig. path to keep the model simpler.

  2. But if your model contain something that in your field have always been found to be sig. and your model, due to other variables, suggested otherwise. They will report the model with the non-sig. path, and then explore what the supposed sig. path is insignificant. Then usually they will run another SEM without that insig. path and compare the fit indices again.

  3. And yes not just the AICs or BICS, even the fit indices (RMSEA, CFI or chi-sq) very often contradict each other, some authors would report only those support what they did (which I dislike), others would compare everything and explain their decision, either it is more 'theory driven' or 'parsimony'.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thanks for your answer! About your 3rd point, I found some papers that suggest which indices to use in what situations (Hu and Bentler 1998, 2009; and Perry et al. 2015). For me these papers have been really useful to justify the choice of different fit indices for different models. $\endgroup$
    – Robbie
    Commented Feb 20, 2015 at 12:41
  • $\begingroup$ @Robbie, Thank you too! What's the full citation for Perry et al 2015? I would like to read that (have read Hu and Bentler)... $\endgroup$
    – ceoec
    Commented Feb 20, 2015 at 13:32
  • 1
    $\begingroup$ The full citation is: Perry, J.L., A.R. Nicholls, and P.J. Clough. 2015. Assessing model fit: caveats and recommendations for confirmatory factor analysis and exploratory structural equation modeling. Meas. Phys. Educ. Exerc. Sci. 19: 12–21. $\endgroup$
    – Robbie
    Commented Feb 21, 2015 at 1:34
0
$\begingroup$

I haven't really done any work related to SEM but I hope my answer helps.

I think there is no golden rule of model selection, people sometimes choose their model by two approaches namely forward selection and backward elimination (three if bidirectional is included). Here are the general steps:

  1. Forward selection

Start fitting the easily model (e.g. no path or some basic paths), then test an additional path using model comparison criterion (AIC, BIC, DIC etc), add (any) paths that improve your model until no more improvement is possible.

  1. Backward elimination (your current approach)

Just the opposite of forward selection. Start the most complicated model (e.g. full model), then test an elimination of a path (usually the non-significant path) using model comparison criterion, drop (any) paths that improve your model until no more improvement can be made.

If you have sufficient time and computing power, you can fit all possible combinations of models then select the one with lowest AIC as you final model. But I have to emphasize that these kinds of approach will not necessarily give you the "correct"/"most appropriate" model, and the criterion like AIC just offer you a tool to get a model in the balance of simplicity and explanatory.

As I know, people sometimes just keep/do not drop the non-significant paths, of which the relations are supported by the theories in their fields e.g. in psychology.

From my point of view, you may report all models that are selected in different selection method together with your full model, and then decide logically which one is the best according to your background knowledge.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thanks for your elaborate answer. The problem with SEM is that forward as well as backward selection do not seem to be appropriate according the theory. SEM basically assumes that one has an a priori defined model that should fit the data. Any changes to this a priori hypothesized model should be as minimal as possible. The problem rises with comparing these adjusted models. There are several measures of model fit which can be compared but also AICs and BICs. A model with a lower AIC doesn't necessarily have to be a better fit when looking at the chi-square, RMSEA or CFI. $\endgroup$
    – Robbie
    Commented Dec 9, 2014 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.