Why would PCA score plots potentially show outliers? If you plot a scree plot of the eigenvalues you can get the “elbow” component. For clarities sake say that a particular data set shows the elbow at four. From this you can plot $PC4$ against $PC3$ and $PC5$ against $PC4$. These two plots may show outliers.
Disregarding the problems of fitting PCA in the presence of outliers, why would these plots potentially show outliers?
 A: 
Disregarding problems of fitting PCA in the presence of outliers, why would these plots potentially show outliers?

It depends on the particular situation but the reason outliers might be visible on a PCA plot is that having an outlier or a few outliers increases the variance in a specific direction. Here is a simplistic 2D illustration:

The outlier in this case is the point in the top right. In this example PCA would look for a linear projection with most variance and the projection on a diagonal (from bottom left towards top right) would have slightly bigger variance compared to a direction parallel to the x or y axis.
And in this example there is another point worth mentioning: since the outlier is affected on all measurements (both x and y axes) almost any projection will have the same point visible as an outlier. So even if outlier doesn't increase the variance by much the PCA lines parallel to x (or y) would still show the outlier standing out from the other points.
As for the reason why outliers could be prone to occur on later PCs: if there is something interesting happening within the data (say you have two classes of observations) then this effect might create more variance compared with the outlier. And so the first PCs will mainly "capture" this kind of variance. And effects producing less variation (like outliers) are left for later projections. Here is an example of that happening:

In this case the two clouds of points produce variance and so first principal component would be parallel to the x axis (dark red line) and after projection the outlier (top right) would not be visible. However second component (orange line) would then pick it up.
A: A PCA is reducing the dimensions of your data. If you reduce your data to 2 or 3 dimensions, that allows you to represent graphically your dataset - the principal componets are your new variables, instead of your original ones.
For example, imagine you had a dataset with three variables A, B and C: if you scaled your dataset (between -1 and 1, for example) and represented that data graphically with your three variables A, B and C in the three axis, you would be able to identify which observations were not similar to the other observations according to those variables - outliers. PCA works in the same way: the dimensions are reduced, your data is scaled, and your new variables are the Principal Components, which allows you to identify observations that are unlike the rest according to those principal components (the outliers).
