I use three examples in my lectures to illustrate what PCA can do (click the links for pointers to the slides). They're chosen to show how useful it is in general data science practice and how powerful it can be (especially given that it's just a linear transformation).
Imagine you're a paleontologist, and you find a shoulder bone. Because of years of training, you recognize immediately that this is a Hominid bone (which is very rare) and not a Chimp bone (which is common).
How do you report that fact? "It's a Hominid bone because I say so", isn't very scientific.
One way would be to use PCA. You measure a bunch of features on the bone and on some similar bones, and you plot the first two principal components. Here's one example of such a plot .
The plot very clearly shows what your trained eyes told you in the first place: the bones of Hominids cluster together, far away from the more common bones of humans, chimps and other apes.
You can even draw a line backwards through the known evolutionary path, to get a hypothesis where humans come from. Turns out chimps and bonobos are a better candidate than gorillas and gibbons.
example 2: DNA
Take about 1300 people in Europe, sequence their DNA, and check about half a million markers. This gives you a dataset with 1300 instances, and 500 thousand features. Apply PCA, and plot it by the first two principal components. Now color the points by where the subject is from. Here's the result .
The plot reveals that the first two principal components provide a blurry picture of the geographical distibution.
I admit it's hard to say what this is useful for, but it certainly illustrates the power of the method.
Take a set of images of people's faces and flatten then into high-dimensional vectors. Run PCA.
Again, you will see clusters for certain meaningful concepts. But here, we can do something else cool. We can take the n-th principal component, and nudge one of the examples in our data a little in that direction.
Here is the result for the first couple of principal components:
1 Fossil hominin shoulders support an African ape-like last common ancestor of humans and chimpanzees. Nathan M. Young, Terence D. Capellini, Neil T. Roach and Zeresenay Alemseged http://www.pnas.org/content/112/38/11829
2 Novembre, J., Johnson, T., Bryc, K., Kutalik, Z., Boyko, A. R., Auton, A., ... & Bustamante, C. D. (2008). Genes mirror geography within Europe. Nature, 456(7218), 98-101. https://www.nature.com/articles/nature07331