Exploratory factor analysis using pooled longitudinal data

Question

I am working on an analysis using exploratory factor analysis (EFA) with the common factor model. This question concerns a methodological issue. Any insights from someone who knows about EFA would be appreciated.

We measured the wellbeing of people using a 15 question survey. The same questionnaire was given the same 100 subjects at 3 different times (there were large gaps in time between when the surveys were administered, therefore we are assuming there was no time dependence [such as carry-over effect]). The goal is to use factor analysis to determine a fixed set of common factors and then calculate the factor scores and to compare the scores of subjects between time points.

Below is my strategy for the analysis. Does this sound reasonable?

Analysis Strategy: Combine all the data as it if were from a single location. That is to say, create a data set with 100 observations from time 1 and append the 100 observations from time point 2. Then append the 100 observations from time point 3. This would give a combined data set with a total of 300 observations.

Using the above data, fit an EFA model using all 15 variables and 300 observations and obtain factors. After computing the factor scores, again treat the data as if it were from 3 different time points (i.e. 100 subjects measured at three time points). Analyze the factors scores with i.e. ANOVA to test if factor scores of a factor change between the time points.

Answer 1

EFA is not the main issue here, you need to think long and hard about the meaning of correlations between your variables/questions.

Your willingness to assume independence is doing all the work and feels like a way to sidestep a difficult problem through wishful thinking. I don't see how it can be reasonable, even with 6 months in between. That would mean assuming that someone's “well-being” is prone to arbitrary changes, with no link to personality or life circumstances because those are changing much more slowly. You can't expect that type of behavior from psychological variables.

Note that personal attributes are not the only trouble here. Assuming independence would also require that changes between the three time points are essentially random with no relation with any other conceivable variable, thus making comparisons useless.

Imagine that you are drawing a different random sample of 100 participants every six months. If you think the population's distribution on your measure could change in the meantime, ratings collected at a given point in time would be more similar to each other than to ratings collected at other times and these correlations would interact with interpersonal differences in unpredictable ways making correlations computed in the way you describe completely uninterpretable.

This aspect of the assumption is not necessarily completely unreasonable (e.g. intelligence or personality are not supposed to change with the season) but you are probably not willing to make it since you do plan to look at changes between time points.

Answer 2

Putting the three time points as three times as many cases does not require assuming independence. Identically replicating three times the measurements obtained at a single time point would essentially produce the same factors as the factor analysis on that single time point. Therefore independence is not a problem. Presenting the three time points as three times as many cases will actually give more variance to those variables evolving across times, which will make the underlying evolving factors more salient. What the analysis of the pooled replicates will do is to work out common factors that inform the fifteen variables, causing them to be correlated. The contribution (weights) of the factors to the variables will be the same (i.e. common) across all measurement times. Once the factor scores are calculated for the 300 'cases', these scores may then be reorganised into three expression occasions for each of the 100 cases and submitted to repeated measure analysis (ANOVA or MANOVA, for instance). If you plan to use the Next Eigenvalue Sufficiency Test (NEST, Achim, 2017) to determine how many common factors are required for your analysis, I recommend applying NEST on the average of the three measurement times, since NEST needs to know the correct number of independent caases.