Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I understand that it's instead correct to cross-validate using new data. Why is it so? It is just that a model will tend to fit the data set that was used to created it better than another randomly sampled set of data?
Could it ever be justified to use the same data for EFA and CFA?
It is generally a bad idea to do an EFA and a CFA on the same data for the exact reason you mention: A factor structure derived from an EFA will almost always fit very well in a CFA using the same data. EFA and CFA are closely related, so it is no surprise that this is the case.
It is common to split data in half and to do EFA on one half and CFA on the other half.
CFA could be justified on the same data as was performed an EFA if you are interested in demonstrating fit indices for comparative purposes, to examine modification indices as a mean to further elaborate on the model, and to examine covariances and such between factors. In these cases, you wouldn't do CFA to demonstrate that your model fits your data or to simply support your theory/model. This would be a very weak test of model fit because its the same data as from the EFA.
