2
$\begingroup$

I have a system of six co-varying variables which, through principle component analysis, can be reduced to two principle components while retaining >95% of variance explained.

The issue with PCA is that the components themselves do not make physical sense within the context of the data they're intended to fit. Some of the entries in the components (but not all, so no NMF) don't "make sense" to have negative weights. To solve this issue, previous work has derived a method for deriving alternative components through a mixture of expert knowledge and simulated derivation data. While they are presented as alternatives to PCA components, these new components do NOT fit the constraints of PCA--they are not orthogonal and they do not maximize explained variance, but they are much more interpretable.

I'm trying to figure out a way to take these components derived from simulated data, and test how well the work on a "real" data set. My first thinking was to try confirmatory factor analysis--by my thinking, these alternative components derived from theory are essentially latent variables that should underlie the observed real data, and using CFA to test that hypothesis should possibly be a valid test to see if that's the case.

However, this brings up two issues: First, this line of reasoning is not an application of factor analysis I've seen anywhere in literature, so I feel shaky on my theoretical footing. Second, I can't find any example application where CFA has been run in the way I'm trying to run it, where I go into the analysis with two pre-computed factor-weight vectors.

I've tried using the sem package in R, with the following model specification

# First components derived from training/expert analysis

f1 -> x1 , NA ,  0.17
f1 -> x2 , NA ,  0.19
f1 -> x3 , NA ,  0.20

# Second components derived from training/expert analysis

f2 -> x1 , NA ,  0.08
f2 -> x2 , NA , -0.03
f2 -> x4 , NA ,  0.20

# Variances -- Not sure about setting latent variances to 1...
f1 <-> f1 , NA , 1 
f2 <-> f2 , NA , 1

x2 <-> x2 , sigma.x2 , NA
x1 <-> x1 , sigma.x1 , NA
x3 <-> x3 , sigma.x3 , NA
x4 <-> x4 , sigma.x4 , NA

When I try to run with this model, I get the following results:

> x.cor<-cor(real.data)
> cfa.model<-specifyModel("sem_model.r")
> cfa.fit<-sem(cfa.model,syn.cor,1500)
Warning message:
In eval(expr, envir, enclos) :
  Could not compute QR decomposition of Hessian.
Optimization probably did not converge.
> summary(cfa.fit)
Error in summary.objectiveML(cfa.fit) : 
  coefficient covariances cannot be computed
In addition: Warning message:
In vcov.sem(object, robust = robust, analytic = analytic.se) :
   singular Hessian: model is probably underidentified.

Am I doing completely the wrong analysis for what I'm trying to test? If not, what am I doing wrong in my implementation? If so, what is an alternative?

$\endgroup$
  • $\begingroup$ A quick estimate of the applicability of your expert variables should be possible by looking at their angle with the plane created the 1st and 2nd PCA components. Regarding your implementation of the CFA... You do not really have much free parameters. You are now only estimating the variance of the $x_i$. What happens if you have the variance or regression coefficients for f1 and f2 as free parameters? $\endgroup$ – Sextus Empiricus Aug 22 '17 at 20:54
2
+25
$\begingroup$

To deal with your questions directly (and then to offer some thoughts on the circumstances surrounding your questions):

Am I doing completely the wrong analysis for what I'm trying to test?

This question is more difficult than it may seem at first. You want a way to test the adequacy of a principal components solution with pre-specified loading values, but component loadings aren't parameter estimates. So while your concern that common factor analyses are different than principal components solutions is well taken, PCA isn't really poised to help you (dis)confirm the adequacy of your model. CFA would give you a way to test the adequacy of your particular model, but as you note, you would have to feel comfortable jumping from component to common factor conceptualizations. For better or worse, in my field this is not all that uncommon--people often use PCA to get an exploratory solution (while actually thinking in terms of common factors) and then move to confirm it via CFA in a separate sample. In your particular case, it sounds like CFA may be the best way for you to address your analytic needs, and switching to a common factor model seems at the very least common, if not entirely defensible.

If not, what am I doing wrong in my implementation?

This part of your question is, I suspect, a bit off-topic for CV, since it's really about your sem syntax. I'm pretty unfamiliar with sem, but in lavaan (which is the package I know), you would want syntax that would look something like below, where you set up a baseline model where your factor loadings are freely estimated (I've just used a simple one-factor three variable example, but I think you'll get the gist), and then compare against it the fit of a model where you constrain the loadings to the particular values you are interested in (both models would have their latent variance fixed to 1, via the std.lv = T option in the model fitting syntax):

baseline = '
f1 =~ x1 + x2 + x3
'

alt.model = '
f1 =~ .17*x1 + .19*x2 + .20*x4
'

anova(baseline.fit, alt.model.fit)

Other Thoughts

I'll say, up front, that what you're proposing is a really strict test; constraining every factor loading to a particular value is likely going to substantially degrade the fit of your model. I'd be really surprised if it was anywhere near acceptable fitting. If you're less concerned about particular value of a loading, and more concerned about the direction of a loading, you might instead consider just looking at the Wald tests of each loading (testing against a null value of zero) as a way of discerning whether your variables load in the direction you intend (if significant, and in correct direction: yes; if significant an in improper direction: no).

I also find the motivating problems/solutions that lead you to this modelling strategy to be a little concerning. Ignoring your original component loadings, for example, begs the question for me: why use the components analysis in the first place if you were intent on forcing a particular structure onto the data? Moreover, how can you be so confident that "expert knowledge" can discern the exact factor loading values in tandem with some simulations? If your analytic strategy was more relaxed, I might feel differently, but again, you're fitting an incredibly restricted--and therefore specific--model; I'm not sure I would ever be inclined to take someone's "expert knowledge" to the bank to the extent of predicting each and every loading value, especially when the data--through your component loadings--are telling me that the expert might be mistaken.

$\endgroup$
0
$\begingroup$

I think there's some confusion about PCA and CFA in this question. PCA summarizes the observed variables into orthogonal components, and does not assume latent variables, nor error variance for the observed variables. CFA assumes latent variables, which may be correlated, and latent error terms.

SEM is typically used to estimate CFA, as it is a latent variable model. I have not seen PCA implemented in SEM, but perhaps that is because my SEM experience is limited to MPlus.

In the SEM framework, I would approach your problem as follows:

  1. Build a CFA model on the training data
  2. Build a CFA model on the testing data
  3. Constrain the parameters of the model from step 2 to be equal to the values of the parameters in step 1
  4. Perform a log-likelihood or chi square test to see whether imposing these constrainst significantly deteriorates model fit
$\endgroup$
  • $\begingroup$ After consulting with the person giving me the components, I have editted the question to more accurately reflect what's going on. $\endgroup$ – Frank Aug 21 '17 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.