Why set weights to 1 in confirmatory factor analysis?

I write this question with reference to an example on p138-142 of the following document: ftp://ftp.software.ibm.com/software/analytics/spss/documentation/amos/20.0/en/Manuals/IBM_SPSS_Amos_User_Guide.pdf.

Here are illustrative figures and a table:

I understand that the latent variable has no natural metric and that setting a factor loading to 1 is done to fix this problem. However, there are a number of things I don't (fully) understand:

1. How does fixing a factor loading to 1 fix this indeterminacy of scale problem?

2. Why fix to 1, rather than some other number?

3. I understand that by fixing one of the factor->indicator regression weights to 1 we thus make all the other regression weights for that factor relative to it. But what happens if we set a particular factor loading to 1 but then it turns out that the higher scores on the factor predict lower scores on the observed variable in question? After we've initially set the factor loading to 1 can we get to a negative understandardized regression weight, or to a negative standardized regression weight?

4. In this context I've seen factor loadings referred to both as regression coefficients and as covariances. Are both these definitions fully correct?

5. Why did we need to fix spatial->visperc and verbal-paragrap both to 1? What would have happened if we just fixed one of those paths to 1?

6. Looking at the standardized coefficient, how can it be that the unstandardized coefficient for wordmean > sentence > paragrap, but looking at the standardized coefficients paragrap > wordmean > sentence. I thought that by fixing paragrap to 1 initially all other variables loaded on the factor were made to be relative to paragrap.

I'll also add in a question which I would imagine has a related answer: why to we fix the regression coefficient for the unique terms (e.g. err_v->visperc) to 1? What would it mean for err_v to have a coefficient of 1 in predicting visperc?

I'd greatly welcome responses even if they do not address all the questions.

• Here's two interesting articles on scale-setting of latent variables: Gonzalez & Griffin (2001): Testing parameters in SEM: Every "one" matters (www-personal.umich.edu/~gonzo/papers/sem.pdf), Little, Slegers, & Card (2006): A non-arbitrary way of identifying and scaling latent variables in SEM and MACS models (agencylab.ku.edu/~agencylab/manuscripts/…) – Patrick Coulombe Mar 29 '14 at 6:09
• What if you set more than 1 weight to one? Are results the same? – Behacad Mar 29 '14 at 12:17

1. Because it then allows you to use the relationship between the latent variable and the observed variable to determine the variance of the latent variable. For example, consider the regression of Y on X. If I am allowed to change the variance of X, say, by multiplying it by a constant, then I can change the regression coefficient arbitrarily. If instead I fix the value of the regression coefficient, then this determines the variance of X.
2. By convention, and to make it easier to compare the coefficients to each other.
3. In that case, the latent variable simply becomes reversed. For example, suppose our latent variable is math ability, our observed variable is the number of errors on a test, and we fix the regression coefficient to 1. Then our latent variable will become "difficulty with math" instead of math ability, and the coefficients for any other observed variables will change accordingly.
4. If the observed variable and the latent variable are both standardized (i.e., standard deviation equal to 1), then the regression coefficient is equal to the covariance.
5. It is fixing spatial -> visperc to 1 that permits estimation of the variance of spatial (see answer to (1) above). Likewise, fixing verbal -> paragrap permits estimation of the variance of verbal. A model with only one of these constraints would not be identifiable.
6. Because the differences between the unstandardized and standardized coefficients depend not only on the variance of verbal, but also on the variances of paragrap, sentence and wordmean. For example, the standardized coefficient for wordmean equals the unstandardized coefficient multiplied by $\frac{SD_{verbal}}{SD_{wordmean}}$, or $2.234 \times \frac{\sqrt{9.682}}{\sqrt{(2.234^2 \times 9.682) + 19.925}} = 0.841$.

Finally, note that err_v is analogous to the error term in a regression model, e.g., $$visperc = \beta_0 + \beta_1 spatial + err\_v$$ We fix the coefficient on err_v (i.e., on the error term) to 1 so that we can estimate the error variance (i.e., the variance of err_v).

1. I may be misunderstanding the phrase "indeterminancy of scale", but I believe it is set to one for identifiability. (That is, the number of unknowns in this system of equations should not exceed the number of equations.) Without setting one of the links to one, there are too many unknowns. Is that the same thing as indeterminancy of scale?

2. In most SEM applications, you are working with covariance matrices, not the raw data. There is an alternative algorithm that uses the original data, called PLS (Partial Least Squares), which might shed some additional light on things for you.

• 1. Most articles have tended to treat indeterminacy of scale and identifiability as though they are separate issues. One argument in favour of the distinction is that if we add in more observed variables then the ratio of knowns to unknowns goes up but that doesn't obviate the need for a loading to get set to 1. 2. Thanks for the tip regarding PLS. – user1205901 - Reinstate Monica May 15 '13 at 2:34
1. Think about the interpretation as if it were just a simple regression. The coefficient reflects the unit difference in the dependent variable associated with a 1 unit difference in the independent variable. Thus if a 1 unit change in in the IV is associated with a 1 unit change in the DV, then the units are funcationally equivalent. You need a unit for the latent variable because you want to estimate its variance, which is not unitless. The identification problem is related, in that for a simple CFA with 1 latent variable and 3 indicators, the model is not identified unless the constraint is made.

2. You can set it to any number, and the overall nature of the results will be the same (easily checked by looking at model fit, which will be identical). It just easier to interpret the model if you set it to 1.

3. Regardless of how you fix any of the factor loadings, you can get positively and negativey loaded items for the same latent variable. You can test this by multiplying one of your indicators by -1 and estimating your model again.

4. They are functionally the same thing if the regression coefficient is unadjusted (i.e. the dependent variable only has 1 arrow pointing to it). If this is the case, one can be calcuated from the other.

5. Try it! Each latent variable needs a scale, for the reasons already stated.

6. This is a scale issue and exactly the reason for using standarized coefficients. I can make any regression coefficient arbitrarily large by dividing the DV by larger and larger numbers. Thus a 1 unit change in the IV will produce larger and larger changes in the units of the DV. By normalizing, and comparing like for like, we avoid this problem.

7. Fixing the error factor loading to 1 just makes interpretation easier. It makes the respective regression equation in the SEM take the familiar form of Y = BX + e (or Y = BX + 1*e).

• I'm confused about what you say in #5 about fixing the covariance to 1. Surely you meant correlation and not covariance (unless both variables have a variance of 1), correct? Also, if you did mean setting the correlation to 1, it seems that the two variables would have effectively been reduced to a single variable (and not merely been put on the same scale), given they would always take the same value – Patrick Coulombe Mar 29 '14 at 5:57

Stata has a very nice documentation on SEM here, look up "Identification 2" section, it has answers to all your questions.

the absence of scale comes because your latent variable is not observable. you may come up with numeric answers in the survey of happiness, but happiness itself is not directly measured. now you have to somehow link the answers such as 1 to 10 to happiness. so you designate one of the questions as an anchor and set its loading to 1.

it doesn't have to be 1, it could be any value, but 1 is convenient.

both spatial and verbal are not observable, so you need to set the scale to both of them, thus you have anchors for each.