0
$\begingroup$

enter image description here

enter image description here

Hi folks,

My path analysis is using primary data that I've collected for a uni project. After removing the very non-signif variables, I have a model (which I hypothesised) that seems to work pretty well. All the relationships are significant and have moderate effect sizes.

I'm just wondering about model fit statistics? So far I believe my chi squared being 2.987 is okay and that fact it is not significant is good? Also CFI and TLI and between 0.9 and 1 which is good. RMSEA however is 0.106 and I am wondering, how bad/acceptable is this? How could I go about lowering it? Many thanks. If anything that could be improved significantly jumps out, please do say.

BTW I tried adding the top modification index to my model and it just made it unidentified.

this is my model, if it at all helps:

model4 <- 'GRAT~a*PBJW
WB~b*GRAT+d*OPT+g*CTRL
OPT~c*PBJW
CTRL~f*PBJW
PBJW~~PBJW
GRAT~~OPT
GRAT~~CTRL
OPT~~CTRL
SIE1:=a*b
SIE2:= c*d
SIE3:=f*g
TE:= SIE1+SIE2+SIE3'
$\endgroup$

1 Answer 1

1
$\begingroup$

The confidence interval around RMSEA includes 0, the minimum, so the estimate of 0.106 does not mean much. (Notice the nonsignificant p-value for the hypothesis that RMSEA is less than or equal to .05.) When sample size is low, RMSEA can be dominated by random sampling error (Rigdon 1996). Moreover, power is very low, partly because your sample size is not large but also because your model has only one degree of freedom, which comes from the lack of a direct relationship between PBJW and WB. MacCallum, Browne & Sugawara (1996) showed the strong connection between degrees of freedom and statistical power.

Adding one more free parameter made DF = 0 and saturated your model. You can't relax any constraint and still have a non-saturated model.

Be careful in interpreting your chi-square. Remember that chi-square only assesses constraints, not the free parameters. So your low power nonsignificant chi-square says there is not enough evidence to reject the hypothesis that there is not a direct path between PBJW and WB, within the context of this model. (That doesn't mean that, within a different model structure, there won't be a direct path between those variables.) But remember also that you have modified your model based on prior results, so you must take any p-values for this version with a grain of salt. No, the field has not built strong procedures for doing that adjustment explicitly, even though models are almost always modified in the course of research projects.

MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological methods, 1(2), 130.

Rigdon, E. E. (1996). CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 3(4), 369-379.

$\endgroup$
2
  • $\begingroup$ thank-you for your insights. Do you mind explaining what 'power' means in this context, as you said it has very low power because of the low DF. Also, regarding the direct path between PBJW and WB, i have tested it in the same model and in a model with the variable CTRL removed, and it has a p value of 0.083, so not signif to the 5% level. That considered, you say that my low power non signif chi square is not enough to rule out that hypothesis. What can I conclude then, if when a direct path between PBJW and BW is included, it isnt signif, and your conclusion? Many thanks. $\endgroup$ Commented Feb 25, 2020 at 13:38
  • $\begingroup$ Power is the probability of detecting a false null hypothesis. Here, the null hypothesis is that the model-implied covariance matrix is equivalent to the population covariance matrix--or, that the excluded path's slope is zero. Low power means that you are unlikely to detect that the null is false, even if it is. So you haven't found anything, but you haven't looked very hard. Low power makes it tough to derive insight. $\endgroup$
    – Ed Rigdon
    Commented Feb 25, 2020 at 17:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.