1
$\begingroup$

I know that in principal component analysis, we can express PCs as a linear combination of the variables, but is it possible to do that for factors in factor analysis?

For my task I am given a dataset:

data SocioEconomics;
input Population School Employment Services HouseValue;
datalines;
5700 12.8 2500 270 25000
1000 10.9 600 10 10000
3400 8.8 1000 10 9000
3800 13.6 1700 140 25000
4000 12.8 1600 140 25000
8200 8.3 2600 60 12000
1200 11.4 400 10 16000
9100 11.5 3300 60 14000
9900 12.5 3400 180 18000
9600 13.7 3600 390 25000
9600 9.6 3300 80 12000
9400 11.4 4000 100 13000
;

I am to do a factor analysis in SAS using the following code:

proc factor data=SocioEconomics n=5 score;
run;

"As each factor/component can be expressed as a linear combination of the standardised observed variables using the above code, answer the following questions;

a) write down the first factor or Factor1 in terms of the standardised variables."

I am not sure if I am lacking the knowledge, but can we express factors in terms of variables?? If anyone can help clarify, that would be great.

This is part of the output:

enter image description here

thanks

$\endgroup$
1
  • $\begingroup$ Possibly something like 0.20*Population + 0.27*School +0.23 *Employment +0.32 *Services +0.27*HouseValue but not really sure. $\endgroup$ Sep 26, 2020 at 8:01

2 Answers 2

1
$\begingroup$

Not strictly (generally speaking), because of the phenomenon of factor indeterminacy. In simple terms, a factor model renders p observed variables into j unobserved common factors + p residual variables, which are also unobserved. There is no unique best way to express the p+j unobserved variables as one best function of the observed variables. Solving, you will find that each common factor must be expressed partly as a function of the model's observed variables and partly as a function of an arbitrary vector. However, with MANY observed variables loading on the factor with STRONG loadings, the share of the common factor due to the observed variables increases, so that the arbitrary part can become small. For details, see, e.g.,

Rigdon, E. E., Becker, J. M., & Sarstedt, M. (2019). Factor indeterminacy as metrological uncertainty: Implications for advancing psychological measurement. Multivariate behavioral research, 54(3), 429-443.

$\endgroup$
1
$\begingroup$

The OP's question interests me on a more general level. In a lot of private studies over years I've done some experiments with a rotational scheme of estimating factors in the sense of CFA-definitions (separation of unique/error-variances). If this is at all meaningful, then also the composition of CFA-factors by items using multiple regression should be meaningful.
Prof. @EdRigdon in his answer pointed out, that one arbitraryness of the CFA-factordetermination lies in the amount of itemspecific error estimated for the model (resulting in variation of CFA-factor estimates), and that this might the OP's intention dubious/not uniquely solvable.

Anyway --- here I show one experimental ansatz to solve for the OP's question only assuming the item-specific variance estimation.
I however do not claim that the following is already a valid/meaningful procedure, because I had never much discussion of my ansatz at all. (Instead I'd like to get feedback whether this proposal is meaningful at all, and in case that some basic misconception is in it, then where it is and possibly how to remove.)


To show the reproducing ability of my ansatz I define a "hidden" factorstructure in normal-distributed data, which has produced some "empirical" dataset of seven items. This "true" factorstructure underlying the empirical dataset should be re-discovered by the following PC/CF-analysis, possibly including final rotations.

The following table shows the true composition of $7$ items by $9$ uncorrelated normaldistributed factors with $n=1000$ measures. $2$ of the factors are common to the items, $7$ of the factors come up as itemspecific- or simply error-variances.
(Remark: I've used this dataset for another discussion, for which a variable $y$ has also been created but is of no significance here)

 udef      uf_1      uf_2   ue_1.1   ue_1.2   ue_1.3   ue_2.1   ue_2.2   ue_2.3     ue_y
-------------------------------------------------------------------------------------------------------------------------------
x_1.1    1.0000    0.1000   0.3000    .        .        .        .        .        .    
x_1.2    0.9000   -0.3000    .       0.2000    .        .        .        .        .    
x_1.3    0.8000    0.2000    .        .       0.3000    .        .        .        .    
x_2.1    0.2000    0.9000    .        .        .       0.4000    .        .        .    
x_2.2    0.4000    0.7000    .        .        .        .       0.2000    .        .    
x_2.3   -0.3000    0.8000    .        .        .        .        .       0.1000    .    
-------------------------------------------------------------------------------------------------------------------------------
    y    0.8000    0.6000    .        .        .        .        .        .       0.4000
-------------------------------------------------------------------------------------------------------------------------------

The resulting correlation-matrix R ("cor" in my program)

    R     x_1.1     x_1.2     x_1.3     x_2.1    x_2.2     x_2.3        y
-------------------------------------------------------------------------------------------------------------------------------
x_1.1    1.0000    0.8556    0.8910    0.2751   0.5395   -0.2438   0.7613
x_1.2    0.8556    1.0000    0.7758   -0.0924   0.1863   -0.6115   0.5171
x_1.3    0.8910    0.7758    1.0000    0.3855   0.6311   -0.1060   0.8042
x_2.1    0.2751   -0.0924    0.3855    1.0000   0.8505    0.7634   0.6467
x_2.2    0.5395    0.1863    0.6311    0.8505   1.0000    0.6158   0.8271
x_2.3   -0.2438   -0.6115   -0.1060    0.7634   0.6158    1.0000   0.2590
-------------------------------------------------------------------------------------------------------------------------------
    y    0.7613    0.5171    0.8042    0.6467   0.8271    0.2590   1.0000
-------------------------------------------------------------------------------------------------------------------------------

The correlation-matrix is cholesky-decomposed into a initially triangular loadingsmatrix L ("lad" in my program) Calling a rotation-routine on L which discerns itemspecific variances for each item and putting the remaining variance into principal factor position:

[85] L2=rot(lad,"ghcu") // leading columns are "common factors", trailing are "individual factors"
[86]       // the method gives individual errors as multiples of SMC-values

|       L2    cf_p1     cf_p2  |   cf_p3     cf_p4     cf_p5     cf_p6  |   err1     err2     err3     err4     err5     err6     err7
|    --------------------------|----------------------------------------|-------------------------------------------------------------
|    x_1.1   0.8647    0.4227  | -0.0293   -0.0229   -0.1169    0.0896  | 0.2247    .        .        .        .        .        .    
|    x_1.2   0.6276    0.7509  | -0.0356   -0.0412   -0.0095   -0.0851  |  .       0.1789    .        .        .        .        .    
|    x_1.3   0.9029    0.2889  | -0.0424    0.1466    0.1040    0.0270  |  .        .       0.2579    .        .        .        .    
|    x_2.1   0.6470   -0.6769  | -0.1685   -0.1005    0.0446   -0.0019  |  .        .        .       0.2876    .        .        .    
|    x_2.2   0.8509   -0.4672  |  0.0321    0.0705   -0.0882   -0.0732  |  .        .        .        .       0.1964    .        .    
|    x_2.3   0.1876   -0.9654  |  0.0339    0.0469   -0.0222    0.0168  |  .        .        .        .        .       0.1695    .    
|    --------------------------|----------------------------------------|-------------------------------------------------------------
|        y   0.9340   -0.0868  |  0.1727   -0.0968    0.0680    0.0127  |  .        .        .        .        .        .       0.2758

We find two remarkable common principal factors .
Of course the estimate for the itemspecific variances are arbitrary (in bounds) which might reference to the comment of Ed Rigdon about factor indeterminacy. I have a couple of estimates implemented; we could for instance try to improve all the following using the Hotelling method to determine the $2$ factors (allowing spurious correlations in the errors which my procedure here avoids) but I've not done this here. Check that the used rotation indeed gives error-estimates proportional to the $(1 − \text{SMC})$-values, showing that assigning $48.33$ % of the $\text{SMR}=(1 − \text{SMC})$-estimates as error-variances leaves the system positive definite (I introduce here the coefficients SMR=1-SMC)

[92]         smr=1 /# diag(inv(cor))               // squared mutiple residuals
             smr = smr  || diag(lad[1..7,7..13]) ^# 2 // concatenated with the 
                                                   // estimates from correlation
[93] smr=smr || (smr[*,2] /# smr[*,1] )   // append ratio as third column

Itemspecific variances estimated by classical **SMC/SMR** method and by my rotation routine "CU" - it actually estimates same "profile" of error variances, only scaled by a constant $\approx 0.4833$ which allows that the the reduced correlation-matrix is left positive semidefinite.

indiv v²      SMR      rot   rot/SMR
-------------------------------------------------------------------------------------------------------------------------------
   x_1.1   0.1045   0.0505    0.4833
   x_1.2   0.0662   0.0320    0.4833
   x_1.3   0.1377   0.0665    0.4833
   x_2.1   0.1711   0.0827    0.4833
   x_2.2   0.0798   0.0386    0.4833
   x_2.3   0.0595   0.0287    0.4833
-------------------------------------------------------------------------------------------------------------------------------
       y   0.1574   0.0761    0.4833
-------------------------------------------------------------------------------------------------------------------------------

To be later able to relate any results of operations to this original loadingsmatrix, we rotate it into a "reference" version which is triangular, and save the used rotation in a rotation-matrix for later inversion.

[96]        t2=gettrans(lad,"drei",1..6,1..6)
[97] L3 = lad * t2

   L3     cf_t1     cf_t2    cf_t3    cf_t4    cf_t5    cf_t6     err1     err2     err3     err4     err5     err6     err7
-------------------------------------------------------------------------------------------------------------------------------
x_1.1    0.9744     .        .        .        .        .       0.2247    .        .        .        .        .        .    
x_1.2    0.8780    0.4439    .        .        .        .        .       0.1789    .        .        .        .        .    
x_1.3    0.9144   -0.0610   0.3060    .        .        .        .        .       0.2579    .        .        .        .    
x_2.1    0.2824   -0.7666   0.2634   0.4250    .        .        .        .        .       0.2876    .        .        .    
x_2.2    0.5536   -0.6755   0.2733   0.2456   0.2522    .        .        .        .        .       0.1964    .        .    
x_2.3   -0.2502   -0.8825   0.2255   0.2310   0.1578   0.0257    .        .        .        .        .       0.1695    .    
-------------------------------------------------------------------------------------------------------------------------------
    y    0.7813   -0.3805   0.2174   0.1817   0.1330   0.2661    .        .        .        .        .        .       0.2758
-------------------------------------------------------------------------------------------------------------------------------

Now the interesting part begins.

1) We locate the first three principal components in the common loadings-space of the first three items $x_{1.*}$ leaving the itemspecific variances untouched. We add one reference-marker item for the first of the just found principal factor, calling that new item $pf_{1.1}$ at the bottom of the matrix:

[100]      l3=rot(l3,"pca",1..3,1..6)
[101]         l3={l3, {1} || null(1,12)}

 CF_P1   cf_p1.1   cf_p1.2   cf_p1.3    cf_t4     cf_t5     cf_t6     err1     err2     err3     err4     err5     err6     err7
-------------------------------------------------------------------------------------------------------------------------------
 x_1.1    0.9599   -0.0737   -0.1508    .         .         .       0.2247    .        .        .        .        .        .    
 x_1.2    0.9246    0.3327    0.0491    .         .         .        .       0.1789    .        .        .        .        .    
 x_1.3    0.9254   -0.2560    0.1073    .         .         .        .        .       0.2579    .        .        .        .    
 x_2.1    0.2034   -0.8241   -0.1272   0.3489   -0.1527   -0.1886    .        .        .       0.2876    .        .        .    
 x_2.2    0.4839   -0.7670   -0.1223   0.2794    0.1471   -0.1555    .        .        .        .       0.1964    .        .    
 x_2.3   -0.3409   -0.8718   -0.1271   0.2507    0.0646   -0.1092    .        .        .        .        .       0.1695    .    
-------------------------------------------------------------------------------------------------------------------------------
     y    0.7419   -0.4949   -0.0847   0.3180    0.0627    0.1281    .        .        .        .        .        .       0.2758
-------------------------------------------------------------------------------------------------------------------------------
pf_1.1    1.0000     .         .        .         .         .        .        .        .        .        .        .        .    
-------------------------------------------------------------------------------------------------------------------------------

2) We locate the first three principal components in the common loadings-space of the second three items $x_{2.*}$ again leaving the itemspecific variances untouched.
We add another reference-marker item for that first principal factor in the new item $pf_{2.1}$ at the bottom of the matrix:

[105]      l3=rot(l3,"pca",4..6,1..6)
[106]         l3={l3, {1} || null(1,12)}

 CF_P2   cf_p2.1   cf_p2.2   cf_p2.3     cf_t1     cf_t2     cf_t3     err1     err2     err3     err4     err5     err6     err7
-------------------------------------------------------------------------------------------------------------------------------
 x_1.1    0.2184    0.9288    0.0066    0.1337    0.1261   -0.0732   0.2247    .        .        .        .        .        .    
 x_1.2   -0.1837    0.9654   -0.0308   -0.0129   -0.0108   -0.0326    .       0.1789    .        .        .        .        .    
 x_1.3    0.3436    0.8673    0.0190    0.1764   -0.1557   -0.0864    .        .       0.2579    .        .        .        .    
 x_2.1    0.9376    0.0770   -0.1799     .         .         .        .        .        .       0.2876    .        .        .    
 x_2.2    0.9004    0.3682    0.1227     .         .         .        .        .        .        .       0.1964    .        .    
 x_2.3    0.8655   -0.4665    0.0672     .         .         .        .        .        .        .        .       0.1695    .    
-------------------------------------------------------------------------------------------------------------------------------
     y    0.6459    0.6502    0.0498   -0.0376    0.0028   -0.2830    .        .        .        .        .        .       0.2758
-------------------------------------------------------------------------------------------------------------------------------
pf_1.1    0.1359    0.9825   -0.0017    0.1062   -0.0126   -0.0685    .        .        .        .        .        .        .    
pf_2.1    1.0000     .         .         .         .         .        .        .        .        .        .        .        .    
-------------------------------------------------------------------------------------------------------------------------------

We can already observe by the correlation between the new marker-variables, that this two principal factors are also slightly oblique/correlated.
Now we determine the composition of the marker-items from the common variance of the 6 $x_{*.*}$ items like in a regression by simply multiplying the first 6 columns by the inverse of the top-left $6 \times 6$ square matrix as new coordinate-system.

This should give now $\beta$-estimates for the composition of the principal factors by the items:

[110] beta = L3[*,1..6] * inv(L3[1..6,1..6])

  BETA   x1.1->p   x1.2->p   x1.3->p   x2.1->p   x2.2->p   x2.3->p
-------------------------------------------------------------------------------------------------------------------------------
 x_1.1    1.0000     .         .         .         .         .    
 x_1.2     .        1.0000     .         .         .         .    
 x_1.3     .         .        1.0000     .         .         .    
 x_2.1     .         .         .        1.0000     .         .    
 x_2.2     .         .         .         .        1.0000     .    
 x_2.3     .         .         .         .         .        1.0000
-------------------------------------------------------------------------------------------------------------------------------
     y    0.5698    7.6312   -0.0874   -1.7658   -5.9612   10.3720
-------------------------------------------------------------------------------------------------------------------------------
pf_1.1    0.3646    0.3512    0.3515     .         .         .    
pf_2.1     .         .         .        0.3844    0.3692    0.3549
-------------------------------------------------------------------------------------------------------------------------------

We see, that the principal factors of the common-variance of the two item-groups are documented as exactly composed by the three items of which they were determined as principal factors by the rotations.

I think this would model what the OP asks for and might be a viable procedure to go.

To generate data for the marker-items $pf_{1.*}$ one can now apply common routines which I won't discuss here.


Additional remark: One would expect, that a varimax-rotation on the two principal factors in the first six items would give a clearer simple-structure and would reproduce a hidden 2-common-factor structure even better:

[131]   vm = rot(l3, "varimax",1..6,1..2)

L2++PC     cf_v1     cf_v2     cf_p3     cf_p4     cf_p5     cf_p6     err1     err2     err3     err4     err5     err6     err7
-------------------------------------------------------------------------------------------------------------------------------
 x_1.1    0.9520    0.1493   -0.0124   -0.1079   -0.0959    0.0005   0.2247    .        .        .        .        .        .    
 x_1.2    0.9445   -0.2563   -0.0469   -0.0216    0.0859    0.0095    .       0.1789    .        .        .        .        .    
 x_1.3    0.9076    0.2817    0.0736    0.1558   -0.0283   -0.0003    .        .       0.2579    .        .        .        .    
 x_2.1    0.1455    0.9278   -0.1842    0.0369    0.0020   -0.0024    .        .        .       0.2876    .        .        .    
 x_2.2    0.4307    0.8706    0.0959   -0.0656    0.0666   -0.0083    .        .        .        .       0.1964    .        .    
 x_2.3   -0.3993    0.8985    0.0629   -0.0116   -0.0173    0.0132    .        .        .        .        .       0.1695    .    
-------------------------------------------------------------------------------------------------------------------------------
     y    0.7052    0.5954    0.0351   -0.0346    0.0238    0.2630    .        .        .        .        .        .       0.2758
-------------------------------------------------------------------------------------------------------------------------------
pf_1.1    0.9978    0.0634    0.0049    0.0078   -0.0148    0.0034    .        .        .        .        .        .        .    
pf_2.1    0.0733    0.9969   -0.0131   -0.0142    0.0192    0.0007    .        .        .        .        .        .        .    
-------------------------------------------------------------------------------------------------------------------------------

This agrees even better with the marker variables for the (oblique) subset-specific principal factors and the "unknown" hidden defining factorstructure.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.