The quotations and plots from the question seem to be extracted from the book "Analysis of Multivariate and High-Dimensional Data" by Koch I, so to answer this question we need to have a look to the related notation that the author uses in it:
- The author uses the word loadings to refer to the eigenvectors of the covariance matrix, $\Sigma$ :
the spectral decomposition of $\Sigma$ is
$$\Sigma = \Gamma\Lambda\Gamma^T \quad\quad\text{ with } \Sigma\,\eta_k = \lambda_k\eta_k \quad\quad\text{ for } k = 1, \cdots,d,$$
where $\Lambda = \text{diag}(\lambda_1, \cdots,\lambda_d )$, with $\lambda_1 > \lambda_2 > · · · > \lambda_d > 0$, and $\Gamma$ is the orthogonal matrix
with columns $\eta_k$ , the eigenvectors of $\Sigma$ corresponding to the eigenvalues $\lambda_k$.
[...]
The vectors $\eta_k$ are sometimes called the loadings as they represent the ‘load’ or ‘weight’
each variable is accorded in the projection
However, note that this use of the word loadings may not be a widespread convention, see e.g. Loadings vs eigenvectors in PCA: when to use one or another?. So this issue may be one of the main difficulties related to the question
- On the other hand, and related to the previous quote:
The vectors $\eta_k$ are sometimes called the loadings as they represent the ‘load’ or ‘weight’
each variable is accorded in the projection
Here we can see that the author uses the word weight to refer to each element of the eigenvector $\eta_k$ as they represent the projection of each variable in the direction of $\eta_k$.
Knowing this we can know answer the questions related to both quoted points:
First point:
- The first eigenvector is concentrated almost exclusively on the largest two variables, with weights of 0.85 (for variable 24) and 0.517 (for variable 4)
As said earlier, these weights represent the projection of each dimension (variable) that model the breast cancer data in the direction of the first eigenvector.
For example, if we wanted to calculate the weight of the first variable in the direction given by the first eigenvector, $\eta_1$, we'd compute the following:
$$ \text{weight}_1^{\{1\}} = \underbrace{(1, 0, \cdots, 0)}_{1\times n}\,\,\underbrace{\eta_1}_{n\times1}$$
Where $n$ is the number of variables that model the breast cancer data. Besides, I've used the upperscript $\{1\}$ and the subscript $1$ to express the chosen variable and eigenvector respectively.
Givent this, using the information given by the first point, we know that:
$$ \begin{align}
\text{weight}_1^{\{24\}}
&= (0,0,\cdots,0,\underbrace{1}_{24\text{th}},0,\cdots,0)\,\,\eta_{1} = 0.85\\
\text{weight}_1^{\{4\}}
&= (0,0,0,1,0,\cdots,0)\,\,\eta_{1} = 0.517
\end{align}$$
Moreover, given that each eigenvector is orthonormal $\Rightarrow \lVert \eta_1 \rVert = 1$, and that $\lVert (\text{weight}_1^{\{24\}}, \text{weight}_1^{\{4\}})^T\rVert = (0.85^2 + 0.517^2)^{0.5} \approx 0.99$. It means that the rest of the weights are smaller $\to$ the first eigenvector is projected (concentrated) mostly in the directions of the $\text{4th}$ and $\text{24th}$ dimensions. Somethig that is also said in the question.
So to sum up, here weights aren't the eigenvalues related to the eigenvectors. The weight of a certain dimension in regards to a certain eigenvector expresses the projection of this dimension in the direction given by the eigenvector.
Second point:
- The second eigenvector is concentrated on the same two variables, but the loadings are interchanged
Given the convention discussed at the beggining that the author uses, I'd argue that the author here uses the word loadings in an incorrect way.
The author uses the word loadings to refer to the eigenvectors $\eta_k$. However the sentence "the loadings are interchanged" would make more sense if we used the word loads (or weights) instead of loadings because we are already talking about the second eigenvector (loading).
Thereby if we follow this reasoning (using the word loads instead of loadings), we know that again, the variables $\text{4th}$ and $\text{24th}$ have the highest weights (or loads) in regards to the second eigenvector $\Rightarrow \eta_2$ is mostly concentrated on these two variables.
The fact that the weights have been interchanged means that now the highest weight corresponds to the $\text{4th}$ variable. Intuitively we can think that this makes sense as both eigenvectors are perpendicular to each other.
Relation with the plots:
As a sidenote, note that this happens because we are dealing with the raw data $\Rightarrow$ As we can see in the first plot, the highest values correspond to the variables that we have been talking about ($\text{4th}$ and $\text{24th}$). Having both of them values with several orders of magnitude bigger than the rest of the variables.
This makes that these two variables dominate the total variability of the model $\to$ As a result the first two eigenvectors (the ones that carry with the largest amount of information/ variability) are mostly concentrated on these two variables.
