3
$\begingroup$

I'm just learning about SEM (specifically basic path analysis models) and wondering if/when a technique like gradient descent is ever used in any of the estimations?

For example, at some point in path analysis you need to get estimates of the model parameters. I believe maximum likelihood is typically used for this, but could gradient descent be used instead? If so, what would be the function you're trying to minimize/maximize?

Later on in the process, once you have your parameters you calculate a model implied covariance matrix and compare it to one from your sample. Does gradient descent step in here as well? Or could it be used instead for both parameter estimation and evaluation of model fit?

I think my confusion stems from having a system of equations for calculating the values in the covariance matrix, and not understanding how gradient descent might handle the solving of those equations to get parameter values?

Improve this question
$\endgroup$
1
  • $\begingroup$ I've never heard of gradient descent being used. $\endgroup$
    – Jeremy Miles
    Commented Sep 16, 2016 at 0:23

1 Answer 1

Reset to default
1
$\begingroup$

I've never heard of gradient descent being used in SEM, but I don't know everything there is to know.

I used Google Scholar to search for the term 'gradient descent' in the journal 'Structural Equation Modeling' and I found zero hits. https://goo.gl/oB7YwI .

Improve this answer
$\endgroup$
3
  • $\begingroup$ yeah I did a similar search and found nothing, but my question is more could it feasibly be used, and if so, how? Rather than has it ever been done. I guess related to that would be what problems occur that may prevent it from being used? $\endgroup$
    – Simon
    Commented Sep 16, 2016 at 0:50
  • $\begingroup$ My hunch would be that it is slow, and that ML works, so why do we need a different method. However, gradient descent is used in IRT, which is closely related (and sometimes equivalent) to SEM. $\endgroup$
    – Jeremy Miles
    Commented Sep 16, 2016 at 2:32
  • 1
    $\begingroup$ Gradient descent would (could) be a method for performing numerical optimization for do maximum likelihood estimation. Gradient descent, in its pure form, is a really lousy algorithm. When performing maximum likelihood estimation, you are trying to maximum the likelihood, or equivalently, the log-likelihood. $\endgroup$
    – Mark L. Stone
    Commented Sep 16, 2016 at 3:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.