To preface, this is my first foray into CFA and SEM, so please forgive me as I am likely making a number of unwitting mistakes.
I am currently working on a study to validate a proposed psychometric attitudinal scale. We have a number of indicator terms obtained from previous literature and from structured interviews (referred to here as
x1 ... x21), and an idea of how these terms are related to a few latent factors (
y1 ... y3).
Here is the lavaan model of those relationships:
cfa.model = " y1 =~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10 + x11 y2 =~ x12 + x13 + x14 + x15 + x16 y3 =~ x17 + x18 + x19 + x20 + x21 "
We currently have a tiny sample of 20 responses and are hoping to move into full data collection in the very near future. However, going into it, we wanted to have an idea--aside from the numerous "rules of thumb" out there (i.e. 10-to-1 participants-to-indicators)--of the sample size required for our analyses. Here is a CSV containing the data, so you have an idea of what I am working with.
Looking into it, it seems like the best approach is CFA with a DWLS or WLSMV estimator, due to the properties of ordinal Likert-type data (Wang & Cunningham 2005, and others).
lavaan is unable to converge using either of these estimators on a data set so small, but does "succeed" using ML, for what it's worth. I am not sure if this is of any use.
So, following Beaujean 2014, I went to the
simsem package to simulate some data and estimate the minimum sample size from there. I thought that using the distributions of the data we already have would be a good idea. However, when I run the following code, I end up with power estimates of 1.000 for everything aside from the factor-factor estimates, which are exceedingly small (< 0.05).
cfa.sim1 = sim(model=cfa.model, n=rep(c(200, 250, 300, 400, 500), 100), lavaanfun="cfa", realData=data, seed=555) summaryParam(cfa.sim1)
plotPower() function shows this doesn't change across the varying sample sizes. It wasn't entirely clear to me what
simsem was using the
realData argument for; whether it uses the distributions of both the indicators and the latent factors according to the model, or just the indicators. I also wasn't clear on why I would be getting 1.000 for all of the power estimates, though I assumed that the small sample size was causing it to fail because there simply wasn't enough information to estimate the distributions. I would be surprised if it were telling me that 200 was a more-than-adequate sample size, considering the number of indicators.
I then thought that, though less ideal, I could give it distributions with a moderate level of skewness (1.5) and kurtosis (3.0) [from Li 2015, "Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares"] and go from there:
cfa.sim.dist = bindDist(p=3, skewness=1.5, kurtosis=3.0) cfa.sim2 = sim(model=cfa.model, n=rep(c(200, 250, 300, 400, 500), 100), lavaanfun="cfa", facDist=cfa.sim.dist, seed=555) summaryParam(cfa.sim2)
However, this still gives me the same power estimates as before (1.000 for all factor-indicator and indicator-indicator, < 0.05 for factor-factor). Do I also need to provide distributions for the indicators via the
indDist argument, or does it generate these from the
So, these are the questions I am currently faced with:
- Is there a way to use the data set I have to estimate the distributions for the simulations and the power analysis, or is it simply too small to be useful?
- If not, what is the best way to proceed? It seemed to fail even when provided with parameters for the factor distributions.
- You may notice from the data set that there is a 'type' variable as well and that this is, in fact, a within-subjects design (each of the 5 participants rated 4 different samples). How do I incorporate this into my CFA model?