Which PCA (or kernel PCA) basis better describes a single test sample? I have two PCA bases obtained by decomposition of two groups of training data. I also have some samples of test data. 
How can I decide which PCA basis fits better each test sample?
I tried to transform each sample to the PCA basis and measure the energy in the off diagonals in the resulting covariance matrix, but I don't think I can actually discuss covariance matrix derived from a single sample.
Edit
After helpful comments, I have the following revised question.
A good method for estimating which is better is to transform the new sample to each of the PCA spaces, truncate the coefficients, transform back and check the reconstruction error.
However, it appears to be more complex in KPCA. I can transform a new sample, $x_{new}$, to the KPCA basis, by introducing a vector $K_{new}=\phi(x_{new})^T\phi(y)$ for all $y$ used for creating the PCA basis and then performing PCA: $p_{new}=K_{new}V$ where $V$ are the eigenvectors for the KPCA.
But after truncating $p_{new}$, reverting back is difficult, since it is governed by the equation $p_{new}=K_{new}V$. It obviously cannot be simply inverted, and using pseudo-inverse for $V$ yields an ill-conditioned problem. Can this be done?
 A: Standard PCA
Usually you would be interested in just several leading PCs, not the whole basis; then you can use reconstruction error of your test sample with each set of PCA axes to decide which set fits your test sample better (the lower the reconstruction error, the better the fit). If you use all PCA axes, then reconstruction error will always be zero, so comparison becomes meaningless. 
"Several leading PCs"  can be as many as you may want or need, but it should be less than the full dimensionality of your space. Setting up some notation, if $\mathbf x$ is your test sample and $\mathbf U_p$ a matrix with $p$ leading principal axes in columns, then the reconstruction error is given by $$L = \|\mathbf x - \mathbf U_p^\vphantom{\top} \mathbf U_p^\top \mathbf x\|^2.$$
In the interest of the following section, let us transform this formula:
\begin{align}
L &= \|\mathbf x\|^2 - 2\mathbf x^\top \mathbf U_p^\vphantom{\top} \mathbf U_p^\top \mathbf x + \mathbf x^\top \mathbf U_p^\vphantom{\top} \mathbf U_p^\top \mathbf U_p^\vphantom{\top} \mathbf U_p^\top \mathbf x
\\&= \|\mathbf x\|^2 - \mathbf x^\top \mathbf U_p^\vphantom{\top} \mathbf U_p^\top \mathbf x
\\ &= \|\mathbf x\|^2 - \|\mathbf U_p^\top \mathbf x\|^2.
\end{align}
This is essentially Pythagoras theorem: squared length of $\mathbf x$ is equal to the sum of the squared length of its projection on the principal space and the
squared length of the reconstruction error (because they are orthogonal).
Kernel PCA
When we do kernel PCA $\left(\mathbf x \mapsto \phi(\mathbf x)\right)$, we cannot compute principal axes $\mathbf U$ because they often live in an infinite-dimensional space. Instead, we apply kernel trick and compute principal components directly, by eigen-decomposition of the kernel matrix $\mathbf K$: if its eigenvectors are $\mathbf V$ and eigenvalues are on the diagonal of $\mathbf S^2$, then kernel principal components (principal components in the target space) are given by $\mathbf S \mathbf V^\top$. Moreover, any new test sample $\mathbf x$ can be projected on the principal space via $\mathbf S^{-1} \mathbf V^\top \mathbf k$, where $\mathbf k$ is a column vector with coordinates $k_i = k(\mathbf x, \mathbf x_i)$ for all training samples $\mathbf x_i$. Here $k(\cdot, \cdot)=\langle \phi(\cdot), \phi(\cdot)\rangle$ is the kernel function. This you know.
Now consider the reconstruction error in the target space:
\begin{align}
\|\phi(\mathbf x) - \mathbf U_p^\vphantom{\top} \mathbf U_p^\top \phi(\mathbf x)\|^2 &= \|\phi(\mathbf x)\|^2 - \|\mathbf U_p^\top \phi(\mathbf x)\|^2
\\ &= k(\mathbf x, \mathbf x) - \|\mathbf S_p^{-1} \mathbf V_p^\top \mathbf k\|^2.
\end{align}
Voilà, this can be computed.
In the context of your question (comparing two sets of kPCA axes for one given $\mathbf x$) the first term is a constant and so the only relevant term is the second, which is simply the variance of the projection onto the first $p$ principal components. So you can take your test sample, project it on the first $p$ components, and look at the variance. The higher the variance, the better the fit.
