2
$\begingroup$

I am reading Eilers and Marx (1996) and at the beginning of page 94 they write, for $Q = B^TWB$, $D$ a symmetric positive definite matrix and $W$ a diagonal matrix,

\begin{align} tr\left(B(Q + D)^{-1}B^TW\right) &= tr\left((I + Q^{-1/2}DQ^{-1/2})^{-1}\right) \\ &=\sum_{i=1}^{n} \frac{1}{1+\epsilon_i} \end{align}

where $\epsilon_i$ are the eigen values of $Q^{-1/2}DQ^{-1/2}$.

My question is, why don't they write: $tr\left(B(Q + D)^{-1}B^TW\right) = tr\left((I + DQ^{-1})^{-1}\right)$?

Then we could calculate $\epsilon_i$ as the ratio of the eigen values of $D$ by the ones of $Q$, based on this answer, right?

I appreciate if anyone one point what I am missing here.

Improve this question
$\endgroup$
4
  • $\begingroup$ While it doesn't answer the question, I think their $D$ is only positive semidefinite: it's a differencing operator that has $k$ zero eigenvalues for degree-$k$ splines $\endgroup$ – Thomas Lumley Jun 5 '20 at 22:55
  • $\begingroup$ Hi @ThomasLumley. I think their $D$ is fact positive semidefinite. The $\mathbf{D}$ I am writting here is what they call $\lambda D^T D$, for $\lambda > 0$, which is positive definite. $\endgroup$ – rcon1 Jun 8 '20 at 11:34
  • $\begingroup$ No, it isn't. At the top of p94 they define $Q_\lambda=\lambdaD^TD$, and the second line after equation (24) starts "Because $k$ eigenvalues of $Q_\lambda$are zero..." $\endgroup$ – Thomas Lumley Jun 8 '20 at 22:10
  • $\begingroup$ @ThomasLumley, you're definitely right! I made a huge confusion here. Thanks a lot! $\endgroup$ – rcon1 Jun 9 '20 at 0:13
1
$\begingroup$

There are two possible explanations

  1. It's not obvious to me (and might well not even be true) that $D$ and $Q^{-1}$ commute. The linked answer was for the case where $DQ^{-1}$ is symmetric and $DQ^{-1}=Q^{-1}D$ which means that the eigenvalues of the product are products of the eigenvalues. If they don't, you can't get the eigenvalues from those of $D$ and $Q$ that way.

  2. It doesn't actually make the next step any easier -- the only fact they need about the $\epsilon_i$ is that $k$ of them are zero. And when they actually compute $\mathrm{tr}(H)$ in the next section of the paper, they do it by adding up the diagonal elements, which is faster than finding an eigendecomposition.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your time. (1.) I didn't note that they had to commute. My fault. I'll check if they commute. (2.) That makes me think, would building matrix $H = B(B^T W B + D)^{-1}B^T W$ and summing up its diagonal entries still be faster than computing the trace as the ratio of the eigen values of $D$ and $Q$ (in case that is valid)? $\endgroup$ – rcon1 Jun 8 '20 at 11:43
  • $\begingroup$ I guess my question wasn't well posed. The main point of my questions is can we write $tr(B(Q+D)^{-1} B^T W)$ as $tr((I + DQ^{-1})^{-1})$ instead of $tr((I + Q^{-1/2} D Q^{-1/2})^{-1})$ for $Q = B^T W B$? $\endgroup$ – rcon1 Jun 8 '20 at 11:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.