Disregarding problems of fitting PCA in the presence of outliers, why would these plots potentially show outliers?
It depends on the particular situation but the reason outliers might be visible on a PCA plot is that having an outlier or a few outliers increases the variance in a specific direction. Here is a simplistic 2D illustration:
The outlier in this case is the point in the top right. In this example PCA would look for a linear projection with most variance and the projection on a diagonal (from bottom left towards top right) would have slightly bigger variance compared to a direction parallel to the x or y axis.
And in this example there is another point worth mentioning: since the outlier is affected on all measurements (both x and y axes) almost any projection will have the same point visible as an outlier. So even if outlier doesn't increase the variance by much the PCA lines parallel to x (or y) would still show the outlier standing out from the other points.
As for the reason why outliers could be prone to occur on later PCs: if there is something interesting happening within the data (say you have two classes of observations) then this effect might create more variance compared with the outlier. And so the first PCs will mainly "capture" this kind of variance. And effects producing less variation (like outliers) are left for later projections. Here is an example of that happening:
In this case the two clouds of points produce variance and so first principal component would be parallel to the x axis (dark red line) and after projection the outlier (top right) would not be visible. However second component (orange line) would then pick it up.