I have the following data setup: $$X \in \mathbb{R}^{n \times d}$$

$$Y \in \{1, ..., K\}^n$$

And I want to find a low-dimensional representation (something like PCA):

$$A^r = embedding(X, Y, r)$$

where $A^r \in \mathbb{R}^{d \times r}$ is an embedding from $d$ to $r$ dimensions, and:

$$A^q = embedding(X, Y, q)$$

where $A^q \in \mathbb{R}^{d \times q}$ where here $r < q \leq d$.

It is often the case when using an embedding algorithm that there is some common subspace the embeddings find, and it is in fact structured such that $A^q_{, 1:r}$ the top $r$ columns of $A^q$ will be such that $A^q_{, 1:r} = A^r$. Ie, in the case of PCA, this would just be due to the Eckart-Young Theorem. This is incredibly common (PCA, among other algs) so much so that I am wondering, is there a name for this situation?


  • $\begingroup$ It sounds like you're asking for the name of the Eckart-Young theorem, but that's meeting you more than half-way. $\endgroup$ – Sycorax says Reinstate Monica Apr 9 '18 at 22:53
  • $\begingroup$ yah, that's exactly what I'm asking about, though not specifically for the frobenius norm. I'm more or less asking if there's some name for an embedding algorithm that follows a generalization of the eckart-young theorem. $\endgroup$ – Eric Apr 9 '18 at 23:22
  • $\begingroup$ In terms of norms, Mirsky generalized the Eckart-Young theorem beyond the Forbenius norm. I'm not aware of a generalization which extends beyond the case of PCA or SVD. $\endgroup$ – Sycorax says Reinstate Monica Apr 9 '18 at 23:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.