Gram matrix

Let $\bf X$ be a n x p dataset with columns (variables) centered. Then p x p $\bf X'X$ is the total scatter matrix while n x n $\bf XX'$ is called sometimes the total Gram matrix $\bf S$ (not to be confused with "a gramian matrix"). The scatter matrix and the Gram matrix share the same nonzero eigenvalues. Values in both are scalar products by nature.

Gram matrix from distances

If we don't know data $\bf X$ but know $\bf D$, the n x n matrix of squared euclidean distances between the data points, we can obtain $\bf S$ from $\bf D$ by easy operation called "double-centering" of the matrix of distances (see). Geometrically it is expressing the distances via scalar products between vectors radiating from the data cloud's centroid (see) and is based on the cosine theorem (see).

Within-group Gram matrix

Consider rows of $\bf X$ (data points) to be partitioned into k groups (and so we have a n x k binary matrix $\bf G$ which is the dummy, indicator variables of the grouping). If we center $\bf X$ within groups (rather than centering by the grand mean): $\bf X_w$, then $\bf X_w'X_w$ is the pooled within-group scatter matrix and $\bf X_wX_w'$ is the within-group Gram matrix $\bf S_w$. Geometrically, $\bf S_w$ is like $\bf S$, only all the group centroids are superposed here, in the grand origin.

Within-group Gram matrix from distances

It is the Question: I can't figure out how to compute $\bf S_w$ from $\bf D$ and $\bf G$. Note that knowing $\bf D$ and $\bf G$ we can get to know the distances between each point and each group centroid, as well as between group centroids (see, see). Still, I feel difficulty to get myself to $\bf S_w$ from all that information. The problem is how to find elegantly the scalar product $s_w$ between two points belonging to different groups.

(Of course, there is a solution via embedding $\bf D$ into Euclidean space by means of Torgerson MDS aka PCoA. This amounts to a construction of data $\bf X$ from $\bf D$, then $\bf S_w$ is found by the usual way. However, I'm after a more direct way from $\bf D$ to $\bf S_w$ than this because PCoA needs eigen- or singular value decomposition.)

Can you help? The Q is a linear algebra Q of multivariate statistics.

  • 1
    $\begingroup$ +1 but this sounds tricky. I thought a bit about it now but could not solve it. $\endgroup$
    – amoeba
    Jul 24, 2018 at 8:56
  • $\begingroup$ Is it true that $S = S_b + S_w$? I think so. Then what you need to compute is $S$ and $S_b$. To get $S$ from $D$ is easy, that's double-centering. And it seems that computing $S_b$ should be somehow doable... At least this way you don't need to deal with scalar products "between two points belonging to different groups" (as you said the main problem is). $\endgroup$
    – amoeba
    Aug 13, 2018 at 12:25
  • 1
    $\begingroup$ @amoeba, somehow doable [from D & G] - how? Note that that n x n "Sb" Gram matrix is a strange matrix - it can easily have some diagonal values negative..., I don't understand the meaning of the matrix. $\endgroup$
    – ttnphns
    Aug 14, 2018 at 15:51


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.