Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?

Question

I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my analysis to also apply kernel PCA. Now, it can be shown that the eigenvalues of $ C $ should be equal to the eigenvalues of the kernel matrix $ \mathbf{K} $: $$ \mathbf{K} \alpha = \lambda \alpha \\ \Leftrightarrow \mathbf{X} \mathbf{X}^T \alpha=\lambda \alpha \\ \Rightarrow \mathbf{X}^T \mathbf{X} \mathbf{X}^T \alpha=\lambda \mathbf{X}^T \alpha \\ \Leftrightarrow Cv=\lambda v $$

With $ \alpha $ being the eigenvector of $ \mathbf{K} $ and $ v:= \mathbf{X}^T \alpha $ being the eigenvector of $ C $.

After applying kernel PCA with a linear kernel (equivalent to "standard" PCA), however, the eigenvalues are not equal. I see, however, a (maybe general) relationship between $ \lambda_{PCA} $ and $ \lambda_{KPCA} $, because $ \overline{\lambda}_{PCA, i} = \overline{\lambda}_{KPCA, i} $ with $ \overline{\lambda}_i = \frac{\lambda_i}{\sum_{k=1}^n \lambda_k} $ for the $ i $-th of the $ n $ eigenvalues.

So why are the eigenvalues not equal? I'm using Python with sklearn.decomposition.PCA and sklearn.decomposition.KernelPCA.

Answer 1

The covariance matrix $ C $ as well as the Gram / kernel matrix $ \mathbf{K} $ have to be scaled by $ \frac{1}{n} $ with $ n $ being the number of samples in the data, assuming the data is centered: $$ C = \frac{1}{n} \mathbf{X}^T \mathbf{X} \\ \mathbf{K} = \frac{1}{n} \mathbf{X} \mathbf{X}^T $$ While the covariance matrix does get scaled in sklearn.decomposition.PCA (actually, PCA is computed via the SVD and the eigenvalues result from $ \lambda_i = \frac{S_i^2}{n-1} $ with $ S $ being the singular values), $ \mathbf{K} $ doesn't get scaled in sklearn.decomposition.KernelPCA. After scaling the eigenvalues of $ \mathbf{K} $ manually by $ \frac{1}{n - 1} $, the descrepancy vanishes completely.