Recently i've started to do some analysis with PCA. I'm trying to seek for variables correlated with a variable X. Should i then visualize and check components where the correlation or contribution of X variable in component is highest? It's not clear for me why everyone always use PC1 and PC2 for the analysis/plots and why they won't examine for example PC3 and PC4? Any help would be highly appreciated!
If your goal were simply dimension reduction, then it would make sense to retain the first (and maybe second) components. The first component accounts for the largest share of the joint variance of the set of variables, while the second components accounts for the second largest share. The hope is that the first and maybe second components may account for the great bulk of variance in the set, turning many predictors into one or two.
If you are primarily interested in variable X, which is part of a large set of variables, it is certainly possible that X will not feature in the first or even second component. Then it could make sense to focus your attention on the component where X does feature. The unrotated components are mutually orthogonal, so you won't find X correlating with variables that don't feature in the same component(s) as X.
You may strike lucky and find that the bulk of X's variance is accounted for by a single component. But if you dump a very large dataset into PCA, you may well be unhappy with the result. If you have some conceptual knowledge about the variables in the dataset, you might use that to shape which variables are included in your PCA with X.