4
$\begingroup$

I want to know how latent variables are estimated in structural equation modeling. I do not understand at what point and in what way the standard SEM software comes up with the estimates for latent variable variances/covariances etc.

To demonstrate, I have simple 3-wave intercept-only panel model based on

$x_{it} = \alpha_{i} + \delta_{it}; \ i = 1,...,n; \ t = 1,2,3$,

the model is trivial, I just chose it because it is the simplest example I could think of. I am uninterested in the means/intercepts, so assume grand mean centered variables.

In structural equation modeling, we achive this by specifying a measurement and structural part of the model on wide-format data (this is the basis for fixed- and random-effects models in SEM, as outlined in Bollen & Brand 2010, but here I am using an alternative notation outlined in Bollen 1989, p.395)

$\eta^* = B\eta^* + \zeta^* \\ y^* = \Lambda \eta^*$

where

$\eta^* = (\alpha, x_1, x_2, x_3)$, $\zeta^* = (\alpha, \delta_1, \delta_2, \delta_3)$, $y^* = (x_1, x_2, x_3)$

$\Lambda = \left[\begin{array}{l}0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{array}\right]$, $B = \left[\begin{array}{l}0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0\end{array}\right]$, $\Psi = \left[\begin{array}{l}\alpha^{2} & 0 & 0 & 0 \\ 0 & \delta_{1}^{2} & 0 & 0 \\ 0 & 0 & \delta_{2}^{2} & 0 \\ 0 & 0 & 0 & \delta_{3}^{2}\end{array}\right]$.

The model-implied covariance matrix is achieved by putting the equation for $\eta^*$ in reduced-form and working out $y^*y^{*\intercal}$, which (if we respect the assumptions reflected in $\Psi$, i.e., no error covariances etc.) leads to

$\begin{align}\Sigma(\theta) & = \Lambda(I - B)^{-1}\Psi(I - B)^{-1 \intercal}\Lambda^{\intercal} \\ & = \left[\begin{array}{c} \alpha^{2} + \delta_{1}^{2} & & \\ \alpha^{2} & \alpha^{2} + \delta_{2}^{2} & \\ \alpha^{2} & \alpha^{2} & \alpha^{2} + \delta_{3}^{2} \end{array}\right] \end{align}$

I understand up to here, and the results are what we should expect: the observed variable at each point in time has just been decomposed into a time-invariant and time-varying error part.

My questions are:

1) What are the parameters in $\theta$, given that the factor loadings/regression coefficients in $B$ have been set to 1.0? Aren't they $\alpha^{2}, \delta_{t}^{2}$?

2) How do I solve for $\alpha^{2}, \delta_{t}^{2}$? Do I set the equation equal to the observed covariance matrix $\Sigma(\theta) = S$, work out the partial derivative with respect to each of the unknowns, set to zero and solve?

$\endgroup$
3
  • $\begingroup$ You're asking a lot of questions here, and it's not easy to follow. You start off talking about SEM, and then start to talk about time invariance - is this longitudinal SEM? A three wave model with a single variable is just identified, so the SEM isn't very interesting. Alpha and delta are not latent variables (they are eta and ksi - sorry, not sure how to put greek letters in comments). The latent variables are not estimated - their variance is estimated, and their mean/intercept is estimated, but the variable doesn't need to exist. $\endgroup$ Nov 28, 2019 at 6:02
  • 1
    $\begingroup$ Try asking more specific questions, and someone is more likely to be able to help. I get lost in this one. $\endgroup$ Nov 28, 2019 at 6:03
  • $\begingroup$ Thank you for the feedback/comments, Jeremy. I have tried to streamline the question, added some possibly important information and have tried to be more specific about my questions. $\endgroup$
    – hendogg87
    Nov 28, 2019 at 13:50

2 Answers 2

2
$\begingroup$

Thanks for the help Jeremy. I found the solution in Bollen (1989), p. 111.

Assume again the three-wave version of the model above, $x_{it} = \alpha_i + \delta_{it}$ and, for the sake of demonstration, assume the observed covariance matrix $S$ is

$\begin{align} S & = \begin{bmatrix} x_{1}^{2} & & \\ x_{2}x_{1} & x_{2}^{2} & \\ x_{3}x_{1} & x_{3}x_{2} & x_{3}^{2} \end{bmatrix} \\ & = \begin{bmatrix} 2.106 & & \\ 1.079 & 1.958 & \\ 1.108 & 1.017 & 2.111 \end{bmatrix} \end{align}$

the model-implied covariance matrix is

$\begin{align} \Sigma(\hat{\theta}) & = \begin{bmatrix} \hat{\phi} + \hat{\psi}_{1} & & \\ \hat{\phi} & \hat{\phi} + \hat{\psi}_{2} & \\ \hat{\phi} & \hat{\phi} & \hat{\phi} + \hat{\psi}_{3} \end{bmatrix} \end{align}$

where $\hat{\phi} = var(\alpha)$ and $\hat{\psi}_{t} = var(\delta_t)$.

We can work out the variance of $\alpha$ with unweighted least squares,

$\begin{align} F_{ULS} & = \frac{1}{2}tr\big((S - \Sigma(\hat{\theta}))^{2}\big) \\ & = \frac{1}{2}tr((x_{1}^{2} - \hat{\phi} + \hat{\psi}_{1})^{2} + (x_{1}x_{2} - \hat{\phi})^{2} + (x_{1}x_{3} - \hat{\phi})^{2} + (x_{2}^{2} - \hat{\phi} + \hat{\psi}_{2})^{2} + (x_{2}x_{3} - \hat{\phi})^{2} + (x_{3}^{2} - \hat{\phi} + \hat{\psi}_{3})^{2}). \end{align}$

We take the first derivative with respect to $\hat{\phi}$ which works out to

$\begin{align} \frac{\partial{F_{ULS}}}{\partial{\hat{\phi}}} & = -x_{1}^{2} + \hat{\phi} + \hat{\psi}_{1} -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}^{2} + \hat{\phi} + \hat{\psi}_{2} -x_{2}x_{3} + \hat{\phi} -x_{3}^{2} + \hat{\phi} + \hat{\psi}_{3}. \end{align}$

Use covariance algebra together with the model assumptions to show

$\begin{align} x_{t}^{2} & = (\alpha + \delta_{t})^{2} \\ & = (\alpha + \delta_{t})(\alpha + \delta_{t}) \\ & = \alpha^{2} + 2\alpha\delta_{t} + \delta_{t}^{2} \\ & = \alpha^{2} + \delta_{t}^{2} \\ & = \phi + \psi_{t} \end{align}$

so $\psi_{t} = x_{t}^{2} - \phi$, which can be substituted into the derivative above

$\begin{align} \frac{\partial{F_{ULS}}}{\partial{\hat{\phi}}} & = -x_{1}^{2} + \hat{\phi} + (x_{1}^{2} - \hat{\phi}) -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}^{2} + \hat{\phi} + (x_{2}^{2} - \hat{\phi}) -x_{2}x_{3} + \hat{\phi} -x_{3}^{2} + \hat{\phi} + (x_{3}^{2} - \hat{\phi}). \end{align}$

which simplifes to

$\begin{align} \frac{\partial{F_{ULS}}}{\partial{\hat{\phi}}} & = -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}x_{3} + \hat{\phi}. \end{align}$

We set this to zero and solve for $\hat{\phi}$, the variance of the latent variable $\alpha$

$\begin{align} 0 & = -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}x_{3} + \hat{\phi} \\ 3\hat{\phi} & = x_{1}x_{2} + x_{1}x_{3} + x_{2}x_{3} \\ 3\hat{\phi} & = 1.079 + 1.108 + 1.017 \\ \hat{\phi} & = \frac{1.079 + 1.108 + 1.017}{3} \\ \hat{\phi} & = 1.068. \end{align}$

Which is just the average covariance on the off-diagonal. I haven't tried this for the ML method, but I assume it should be straightforward. The results can also be confirmed in lavaan, see code below.

# Packages
install.packages("lavaan")
library(lavaan)
# Observed covariance matrix
S <- matrix(c(2.106, 1.079, 1.108, 
              1.079, 1.958, 1.017, 
              1.108, 1.017, 2.111), nrow = 3, ncol = 3)
# Names 
colnames(S) <- c("x1", "x2", "x3")
rownames(S) <- c("x1", "x2", "x3")
# Lavaan model 
m1 <- '
a =~ 1*x1 + 1*x2 + 1*x3
' 
m1.fit <- sem(m1, sample.cov = S, sample.nobs = 1000)
summary( m1.fit)
$\endgroup$
2
  • $\begingroup$ Thanks for posting! I understand now. :) $\endgroup$ Dec 2, 2019 at 17:39
  • 1
    $\begingroup$ Typical somehow: have to know the answer before you can properly ask the question! Thanks for pointing me in the right direction! $\endgroup$
    – hendogg87
    Dec 2, 2019 at 18:06
1
$\begingroup$

Your model isn't clear to me, that's not necessarily bad, but I think you can express it more simply.

You don't need $B$ and $\Psi$ or $\alpha$ in your equations. These aren't necessary for the simple model you specify. You can ignore $\delta$ for this model, if you want to keep things simple and just standardize everything.

If you do this, then the implied correlation matrix is given by:

$\Sigma(\Theta) = \Lambda \Lambda ^T$

The sample correlation matrix is S, and you fit the model by minimizng $F_{ML}$:3

$F_{ML} = log(det(S)) + tr(S \Sigma(\Theta^{-1}) - log(det(\Sigma(\Theta)) - p$

Where det is the determinant, and tr is the trace of the matrix, and p is the number of parameters estimated.

This is the simplest case, other more complex cases extend this, but the basic idea is the same: Find the implied covariance matrix, given the parameters. Minimize F.

I hope this is what you were asking. :) If not, feel free to ask for clarification.

(I wrote a paper about this once: https://www.researchgate.net/publication/7151927_Confirmatory_factor_analysis_using_Microsoft_Excel)

$\endgroup$
4
  • $\begingroup$ Oh, sorry, I realize you were using alpha differently to the way I extended. Ignore that part. $\endgroup$ Nov 28, 2019 at 19:05
  • $\begingroup$ Thanks again Jeremy. I don't think it is possible to ignore the other matrices, since in this case Lambda*Lambda' (I don't know either how to add math to comments) results in a 3x3 identitiy matrix so there are no parameters that could be manipulated to try an minimize F_ML. But I think this is my fault for using an unintuitive example. Maybe my main question with regards to your answer is: what would you say are the parameters in Theta in your description (Sigma(Theta))? $\endgroup$
    – hendogg87
    Nov 29, 2019 at 12:15
  • $\begingroup$ I still think they must be alpha^2 and delta_t^2. And I know there is an explicit solution to each because I can work it out with least squares (the part I deleted from the original question). So my intuition is that there must be a way to solve either Sigma(theta) or F_ML to give me the variances of alpha and delta (the between- and within-person variance, respectively). $\endgroup$
    – hendogg87
    Nov 29, 2019 at 12:26
  • $\begingroup$ To avoid an extended discussion here, I created a chat room. chat.stackexchange.com/rooms/101673/… $\endgroup$ Dec 1, 2019 at 6:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.