1
$\begingroup$

I have this confusion related to minimization of gaussian likelihood function. The negative of the log likelihood of gaussian distribution is

$-\log \det(Q) + \text{tr}(SQ) + \lambda||Q||_{1}$ where $Q$ is the precision matrix to be estimated

I am following this paper

It's been mentioned that

Using sub-gradient notation, we can write the optimality conditions (aka “normal equations”) for a solution to the above equation as

$-Q^{-1} + S + \lambda\Gamma = 0$

where $\Gamma_{jk} = \text{sign}(Q_{jk})$ if $Q_{jk} \neq 0$ and

$\Gamma_{jk} \in [-1, 1]$ if $Q_{jk} = 0$

I didn't get how come

$\Gamma_{jk} \in [-1, 1]$ if $Q_{jk} = 0$

Can anyone please explain it should be 0 isn't it. I didn't get how the interval $[-1, 1]$ came

Suggestions?

$\endgroup$

1 Answer 1

1
$\begingroup$

The equation says that, at the minimum, one of the subgradients of $$-\log \det Q + \text{tr}( S Q ) + \lambda \| Q \|_1$$ is zero.

The last term, $\lambda \Gamma$ is a subgradient of $\lambda \left\| Q \right\|_1$. Since $\left\| Q \right\|_1 = \sum_{ij} \left| Q_{ij} \right|$, we just have to find the subgradients of the 1-dimensional function $f(q) = \left| q \right|$. If $q \neq 0$, $f$ is differentiable, and there is only one subgradient: $f'(q) = \text{sign}\,q$. For $q=0$, any number in $[-1,1]$ is a subgradient.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.