0
$\begingroup$

So, suppose I have an objective function $\mathcal{L}(\Sigma)$ where $\Sigma$ is a positive definite matrix. Now, I want to optimize this function using gradient descent. Now, I think if I compute the gradient and write the update equation for $\Sigma$, the positive definite constraint may not be maintained. How can I be sure that the optimum I get is also at a pd matrix.

$\endgroup$

2 Answers 2

1
$\begingroup$

This problem occurs prominently in for instance estimation of mixed models. The R packages nlme and lme4 and certainly others use the log-Cholesky parameterization of the positive definite covariance matrix(ices).

For details see Difference between Cholesky decomposition and log-cholesky Decomposition

$\endgroup$
1
$\begingroup$

All symmetric positive definite matrix are diagonalizable and have all eigenvalues positive.
The matrix logarithm will be a symmetric matrix.
If the matrix is size 3 for example, the logarithm will have 6 free parameters; in general $n(n+1)/2$ parameters are needed for the logarithm. Define the function to optimize over these parameters for the logarithm of the matrix, then use the matrix exponential to find the corresponding positive definite matrix you use in the objective function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.