Integrating length for input-space feature PC projections in kernel PCA

Question

I read a paper detailing the algebraic process of kernel PCA. I have question though: the paper details the projection of new points onto the new eigenvectors in the feature space, but what if I want eigenvectors themselves? I am performing an analysis that compares eigenvector magnitudes in a Gaussian inner product space.

The closest I found in the paper was the definition of the eigenvectors as a hypothetical linear combination of data vectors in the kernel space:

$$\mathbf{V} = \sum_{i = 1}^l \alpha_i \Phi\left(\mathbf{x}_i\right).$$

From the paper, it seems that you derive all such $\alpha$ with the relation

$$\ell\lambda\mathbf{\alpha}=K\mathbf{\alpha}$$ (where $\mathbf{\alpha}$ is supposed to look bold as a vector).

But even after you obtain $\alpha_i$, you can't compute the eigenvectors in the feature space from the previous formula because, in the case of a Gaussian kernel that operates on $\|\mathbf{x} - \mathbf{y}\|$, there are infinitely many dimensions in the the "hidden" range of $\Phi$.

How can I compute $\mathbf{V}$, the eigenvectors in the $\mathbf{\Phi}$?

---

Edit 0

I understand that we can't formalize eigenvectors in a infinite-dimensional space implied by a Gaussian-norm inner product kernel. What I actually want is the length of the eigenvector. In the linear case, that would be the semimajor axes of the imaginary ellipse. I want to be able to have, for a "classification" PCA case as this one, a graph like the one on the left showing a curved (for lack of a better word) eigenvector projected down to the input space. I'd prefer a parametric form over which I may compute an arclength integral.

---

Edit 1

Oops. I meant principal component, not eigenvector.

Answer 1

[I apologize that I have not posted this answer earlier, as promised.]

This question seems to have been built upon series of confusions (partly clarified in the comments above and reflected in the edits).

Q1: How can one obtain eigenvectors [of the covariance matrix] in the target space?

This is generally not possible. For some kernels, like e.g. Gaussian kernel, the target space is infinite-dimensional. There is no way one can obtain an eigenvector there: it would need to have infinitely many coordinates! The beauty of the "kernel trick" is that it allows to compute projections of the data onto this eigenvector without ever computing the eigenvector itself.

Q2: Okay, but can one at least compute the length of an eigenvector?

Yes. The length is equal to $1$, because that's how we choose to normalize eigenvectors.

Q3: But what about the length of an eigenvector, like it's depicted on 2D examples of PCA?

This length is simply the square root of the respective eigenvalue. They are directly computed in kPCA.

---

Having said all that, here is my best guess about what you actually wanted to get.

I want to be able to have, for a "classification" PCA case as this one, a graph like the one on the left showing a curved (for lack of a better word) eigenvector projected down to the input space. I'd prefer a parametric form over which I may compute an arclength integral.

It seems you want to get the length of the decision boundary between two classes in the original feature space. The main problem here is that kPCA is not a classification technique. There is no way kPCA will give you a decision boundary: it's simply projects all your data to some subspace of the target space. If you are lucky, the classes will be linearly separated. But if you need a decision boundary, you will need to use some other, additional, approach to obtain it, and it should be a supervised algorithm (kPCA is unsupervised).

After you come up with or choose an algorithm, then you can ask yourself how to compute the length of the decision boundary. E.g. perhaps you can use some linear classifier (such as LDA, logistic regression, or linear SVM) to get a linear boundary in the kPCA space, and then ask what is the length of the corresponding boundary in the original space. It is not going to be very easy, as far as I can see.