Principal Components Analysis (PCA) doesn't select among the original features unless the features are orthogonal. It lowers the dimension of the feature space by ignoring PCs (linear combinations of potentially all the predictors) that explain little of the feature-set variance, but in general it does not remove any of the original features from the problem.
Some find the ability of LASSO to choose among several correlated features to be a strength rather than a weakness, particularly when the number of features greatly exceeds the number of cases. Yes, LASSO's choice among correlated features can be highly sample dependent. Nevertheless, as LASSO penalizes the coefficients for the retained variables it can still be useful for developing prediction models. The "disadvantage" is when one tries to interpret the selection by LASSO in terms of "variable importance" for inference.
The issues raised in this question point out a potential advantage of ridge regression, which is effectively PCA with varying weights among the PCs rather than simple 0/1 weights. (See page 79 of ESLII.) Try both LASSO and ridge regression on multiple bootstrap samples of a data set having correlated predictors. The particular features selected by LASSO may change dramatically among the bootstrap samples. This illustrates the mistake of interpreting choices by LASSO in terms of an underlying true "variable importance." The coefficients of correlated predictors in ridge regression may be variable among bootstrap samples but the ridge models will contain all the features.