4
$\begingroup$

I want to know how latent variables are estimated in structural equation modeling. I do not understand at what point and in what way the standard SEM software comes up with the estimates for latent variable variances/covariances etc.

To demonstrate, I have simple 3-wave intercept-only panel model based on

$x_{it} = \alpha_{i} + \delta_{it}; \ i = 1,...,n; \ t = 1,2,3$,

the model is trivial, I just chose it because it is the simplest example I could think of. I am uninterested in the means/intercepts, so assume grand mean centered variables.

In structural equation modeling, we achive this by specifying a measurement and structural part of the model on wide-format data (this is the basis for fixed- and random-effects models in SEM, as outlined in Bollen & Brand 2010, but here I am using an alternative notation outlined in Bollen 1989, p.395)

$\eta^* = B\eta^* + \zeta^* \\ y^* = \Lambda \eta^*$

where

$\eta^* = (\alpha, x_1, x_2, x_3)$, $\zeta^* = (\alpha, \delta_1, \delta_2, \delta_3)$, $y^* = (x_1, x_2, x_3)$

$\Lambda = \left[\begin{array}{l}0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{array}\right]$, $B = \left[\begin{array}{l}0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0\end{array}\right]$, $\Psi = \left[\begin{array}{l}\alpha^{2} & 0 & 0 & 0 \\ 0 & \delta_{1}^{2} & 0 & 0 \\ 0 & 0 & \delta_{2}^{2} & 0 \\ 0 & 0 & 0 & \delta_{3}^{2}\end{array}\right]$.

The model-implied covariance matrix is achieved by putting the equation for $\eta^*$ in reduced-form and working out $y^*y^{*\intercal}$, which (if we respect the assumptions reflected in $\Psi$, i.e., no error covariances etc.) leads to

$\begin{align}\Sigma(\theta) & = \Lambda(I - B)^{-1}\Psi(I - B)^{-1 \intercal}\Lambda^{\intercal} \\ & = \left[\begin{array}{c} \alpha^{2} + \delta_{1}^{2} & & \\ \alpha^{2} & \alpha^{2} + \delta_{2}^{2} & \\ \alpha^{2} & \alpha^{2} & \alpha^{2} + \delta_{3}^{2} \end{array}\right] \end{align}$

I understand up to here, and the results are what we should expect: the observed variable at each point in time has just been decomposed into a time-invariant and time-varying error part.

My questions are:

1) What are the parameters in $\theta$, given that the factor loadings/regression coefficients in $B$ have been set to 1.0? Aren't they $\alpha^{2}, \delta_{t}^{2}$?

2) How do I solve for $\alpha^{2}, \delta_{t}^{2}$? Do I set the equation equal to the observed covariance matrix $\Sigma(\theta) = S$, work out the partial derivative with respect to each of the unknowns, set to zero and solve?

| cite | improve this question | | | | |
$\endgroup$
  • $\begingroup$ You're asking a lot of questions here, and it's not easy to follow. You start off talking about SEM, and then start to talk about time invariance - is this longitudinal SEM? A three wave model with a single variable is just identified, so the SEM isn't very interesting. Alpha and delta are not latent variables (they are eta and ksi - sorry, not sure how to put greek letters in comments). The latent variables are not estimated - their variance is estimated, and their mean/intercept is estimated, but the variable doesn't need to exist. $\endgroup$ – Jeremy Miles Nov 28 '19 at 6:02
  • 1
    $\begingroup$ Try asking more specific questions, and someone is more likely to be able to help. I get lost in this one. $\endgroup$ – Jeremy Miles Nov 28 '19 at 6:03
  • $\begingroup$ Thank you for the feedback/comments, Jeremy. I have tried to streamline the question, added some possibly important information and have tried to be more specific about my questions. $\endgroup$ – hendogg87 Nov 28 '19 at 13:50
2
$\begingroup$

Thanks for the help Jeremy. I found the solution in Bollen (1989), p. 111.

Assume again the three-wave version of the model above, $x_{it} = \alpha_i + \delta_{it}$ and, for the sake of demonstration, assume the observed covariance matrix $S$ is

$\begin{align} S & = \begin{bmatrix} x_{1}^{2} & & \\ x_{2}x_{1} & x_{2}^{2} & \\ x_{3}x_{1} & x_{3}x_{2} & x_{3}^{2} \end{bmatrix} \\ & = \begin{bmatrix} 2.106 & & \\ 1.079 & 1.958 & \\ 1.108 & 1.017 & 2.111 \end{bmatrix} \end{align}$

the model-implied covariance matrix is

$\begin{align} \Sigma(\hat{\theta}) & = \begin{bmatrix} \hat{\phi} + \hat{\psi}_{1} & & \\ \hat{\phi} & \hat{\phi} + \hat{\psi}_{2} & \\ \hat{\phi} & \hat{\phi} & \hat{\phi} + \hat{\psi}_{3} \end{bmatrix} \end{align}$

where $\hat{\phi} = var(\alpha)$ and $\hat{\psi}_{t} = var(\delta_t)$.

We can work out the variance of $\alpha$ with unweighted least squares,

$\begin{align} F_{ULS} & = \frac{1}{2}tr\big((S - \Sigma(\hat{\theta}))^{2}\big) \\ & = \frac{1}{2}tr((x_{1}^{2} - \hat{\phi} + \hat{\psi}_{1})^{2} + (x_{1}x_{2} - \hat{\phi})^{2} + (x_{1}x_{3} - \hat{\phi})^{2} + (x_{2}^{2} - \hat{\phi} + \hat{\psi}_{2})^{2} + (x_{2}x_{3} - \hat{\phi})^{2} + (x_{3}^{2} - \hat{\phi} + \hat{\psi}_{3})^{2}). \end{align}$

We take the first derivative with respect to $\hat{\phi}$ which works out to

$\begin{align} \frac{\partial{F_{ULS}}}{\partial{\hat{\phi}}} & = -x_{1}^{2} + \hat{\phi} + \hat{\psi}_{1} -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}^{2} + \hat{\phi} + \hat{\psi}_{2} -x_{2}x_{3} + \hat{\phi} -x_{3}^{2} + \hat{\phi} + \hat{\psi}_{3}. \end{align}$

Use covariance algebra together with the model assumptions to show

$\begin{align} x_{t}^{2} & = (\alpha + \delta_{t})^{2} \\ & = (\alpha + \delta_{t})(\alpha + \delta_{t}) \\ & = \alpha^{2} + 2\alpha\delta_{t} + \delta_{t}^{2} \\ & = \alpha^{2} + \delta_{t}^{2} \\ & = \phi + \psi_{t} \end{align}$

so $\psi_{t} = x_{t}^{2} - \phi$, which can be substituted into the derivative above

$\begin{align} \frac{\partial{F_{ULS}}}{\partial{\hat{\phi}}} & = -x_{1}^{2} + \hat{\phi} + (x_{1}^{2} - \hat{\phi}) -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}^{2} + \hat{\phi} + (x_{2}^{2} - \hat{\phi}) -x_{2}x_{3} + \hat{\phi} -x_{3}^{2} + \hat{\phi} + (x_{3}^{2} - \hat{\phi}). \end{align}$

which simplifes to

$\begin{align} \frac{\partial{F_{ULS}}}{\partial{\hat{\phi}}} & = -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}x_{3} + \hat{\phi}. \end{align}$

We set this to zero and solve for $\hat{\phi}$, the variance of the latent variable $\alpha$

$\begin{align} 0 & = -x_{1}x_{2} + \hat{\phi} -x_{1}x_{3} + \hat{\phi} -x_{2}x_{3} + \hat{\phi} \\ 3\hat{\phi} & = x_{1}x_{2} + x_{1}x_{3} + x_{2}x_{3} \\ 3\hat{\phi} & = 1.079 + 1.108 + 1.017 \\ \hat{\phi} & = \frac{1.079 + 1.108 + 1.017}{3} \\ \hat{\phi} & = 1.068. \end{align}$

Which is just the average covariance on the off-diagonal. I haven't tried this for the ML method, but I assume it should be straightforward. The results can also be confirmed in lavaan, see code below.

# Packages
install.packages("lavaan")
library(lavaan)
# Observed covariance matrix
S <- matrix(c(2.106, 1.079, 1.108, 
              1.079, 1.958, 1.017, 
              1.108, 1.017, 2.111), nrow = 3, ncol = 3)
# Names 
colnames(S) <- c("x1", "x2", "x3")
rownames(S) <- c("x1", "x2", "x3")
# Lavaan model 
m1 <- '
a =~ 1*x1 + 1*x2 + 1*x3
' 
m1.fit <- sem(m1, sample.cov = S, sample.nobs = 1000)
summary( m1.fit)
| cite | improve this answer | | | | |
$\endgroup$
  • $\begingroup$ Thanks for posting! I understand now. :) $\endgroup$ – Jeremy Miles Dec 2 '19 at 17:39
  • 1
    $\begingroup$ Typical somehow: have to know the answer before you can properly ask the question! Thanks for pointing me in the right direction! $\endgroup$ – hendogg87 Dec 2 '19 at 18:06
1
$\begingroup$

Your model isn't clear to me, that's not necessarily bad, but I think you can express it more simply.

You don't need $B$ and $\Psi$ or $\alpha$ in your equations. These aren't necessary for the simple model you specify. You can ignore $\delta$ for this model, if you want to keep things simple and just standardize everything.

If you do this, then the implied correlation matrix is given by:

$\Sigma(\Theta) = \Lambda \Lambda ^T$

The sample correlation matrix is S, and you fit the model by minimizng $F_{ML}$:3

$F_{ML} = log(det(S)) + tr(S \Sigma(\Theta^{-1}) - log(det(\Sigma(\Theta)) - p$

Where det is the determinant, and tr is the trace of the matrix, and p is the number of parameters estimated.

This is the simplest case, other more complex cases extend this, but the basic idea is the same: Find the implied covariance matrix, given the parameters. Minimize F.

I hope this is what you were asking. :) If not, feel free to ask for clarification.

(I wrote a paper about this once: https://www.researchgate.net/publication/7151927_Confirmatory_factor_analysis_using_Microsoft_Excel)

| cite | improve this answer | | | | |
$\endgroup$
  • $\begingroup$ Oh, sorry, I realize you were using alpha differently to the way I extended. Ignore that part. $\endgroup$ – Jeremy Miles Nov 28 '19 at 19:05
  • $\begingroup$ Thanks again Jeremy. I don't think it is possible to ignore the other matrices, since in this case Lambda*Lambda' (I don't know either how to add math to comments) results in a 3x3 identitiy matrix so there are no parameters that could be manipulated to try an minimize F_ML. But I think this is my fault for using an unintuitive example. Maybe my main question with regards to your answer is: what would you say are the parameters in Theta in your description (Sigma(Theta))? $\endgroup$ – hendogg87 Nov 29 '19 at 12:15
  • $\begingroup$ I still think they must be alpha^2 and delta_t^2. And I know there is an explicit solution to each because I can work it out with least squares (the part I deleted from the original question). So my intuition is that there must be a way to solve either Sigma(theta) or F_ML to give me the variances of alpha and delta (the between- and within-person variance, respectively). $\endgroup$ – hendogg87 Nov 29 '19 at 12:26
  • $\begingroup$ To avoid an extended discussion here, I created a chat room. chat.stackexchange.com/rooms/101673/… $\endgroup$ – Jeremy Miles Dec 1 '19 at 6:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.