show that minimising the distance to approximated vector is equivalent to selecting features with the most variance

Question

First off, a little disclaimer: I am basing the question on my own interpretation of the problem. It's well possible that I am mistaken.

Setup: We are given: feature vectors $x_1,\ldots, x_k$ with dimension $N$ such that $k \geq N$ and $\sum_{i=1}^k x_i=0$; $N$ orthonormal basis vectors $v_i$ of dimension $N$. We define approximated vectors $x'$ such that $x'_j = \sum_{i}^d v_i^T x_j v_i$ where $j = 1,\ldots ,k$ and $0 < d < N$.

Problem: Show that the following two methods are equivalent.

1. minimize the objective function $\sum_{i}^K \|x_i - x'_i\|^2$
2. maximize $\sum_{i}^d v_i^T S v_i$ where $S$ is the covariance matrix of the vectors $x$

Question: My understanding is that method 1 minimizes the distance between vector $x$ and its approximation $x'$, and that method 2 selects the features with the highest (co)variance. As I am aware, the eigenvector of the covariance matrix with the highest eigenvalue indicates the axis of highest variance, and that, by implication, a line along that axis would be the Least Squares approximator for the data. Therefore, I see how the methods are equivalent in theory, but I'm having trouble finding a mathematical/formal representation of this intuition based on the given expressions.

Answer 1

at least for the trivial case, where the vectors v are rows of the unit Matrix, we can say that method 1 selects the dimensions of x where the elements have the largest absolute values. additionally assuming that the data was centered (i.e. mean = 0), and considering the definition of variance $ \frac 1n \sum(x - \mu)^2$ we can see that method 2 also selects the dimensions where elements have the largest absolute values