# Embedding Dimension finds same projection direction with more projection dimensions

I have the following data setup: $$X \in \mathbb{R}^{n \times d}$$

$$Y \in \{1, ..., K\}^n$$

And I want to find a low-dimensional representation (something like PCA):

$$A^r = embedding(X, Y, r)$$

where $A^r \in \mathbb{R}^{d \times r}$ is an embedding from $d$ to $r$ dimensions, and:

$$A^q = embedding(X, Y, q)$$

where $A^q \in \mathbb{R}^{d \times q}$ where here $r < q \leq d$.

It is often the case when using an embedding algorithm that there is some common subspace the embeddings find, and it is in fact structured such that $A^q_{, 1:r}$ the top $r$ columns of $A^q$ will be such that $A^q_{, 1:r} = A^r$. Ie, in the case of PCA, this would just be due to the Eckart-Young Theorem. This is incredibly common (PCA, among other algs) so much so that I am wondering, is there a name for this situation?

Thanks!

• It sounds like you're asking for the name of the Eckart-Young theorem, but that's meeting you more than half-way. – Sycorax says Reinstate Monica Apr 9 '18 at 22:53
• yah, that's exactly what I'm asking about, though not specifically for the frobenius norm. I'm more or less asking if there's some name for an embedding algorithm that follows a generalization of the eckart-young theorem. – Eric Apr 9 '18 at 23:22
• In terms of norms, Mirsky generalized the Eckart-Young theorem beyond the Forbenius norm. I'm not aware of a generalization which extends beyond the case of PCA or SVD. – Sycorax says Reinstate Monica Apr 9 '18 at 23:57