5
$\begingroup$

I've read a lot about how PCA is used to reduce the dimensionality of data, including this great answer and this mathy post. But I'm unclear what this does when applied to image data?

For example, given a set of vectors from images of faces, I could reduce the dimensions:

pca = PCA(n_components=100, svd_solver='randomized', whiten=True)
pca.fit(faces)
compressed = pca.transform(faces)

But when I look at the results, I see nothing like an image:

example = compressed[0]
example *= 255.0
example = example.astype('uint8')
example = example.reshape((10,10))
output = Image.fromarray(example)
output = output.resize((200,200))
output.save('ExamplePCA.jpg')

example of pca to image

Can someone describe for me (someone with limited matrix math/linear algebra knowledge) what is being represented here?

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ have you seen this answer? stats.stackexchange.com/questions/177102/… $\endgroup$
    – jld
    Mar 29, 2018 at 5:11
  • $\begingroup$ @Chaconne – is PCA similar? From the answer here below, it sounds like there is no visual correlation between the PCA space and the original image. $\endgroup$
    – JeffThompson
    Mar 29, 2018 at 12:18
  • 1
    $\begingroup$ ah I misread your question, never mind $\endgroup$
    – jld
    Mar 29, 2018 at 13:13
  • 1
    $\begingroup$ @JeffThompson Look into ZCA ("Zero-phase component analysis"), there's a good question on it that also helps to understand PCA on image collections. $\endgroup$
    – Firebug
    Mar 29, 2018 at 16:41

1 Answer 1

Reset to default
4
$\begingroup$

The transformed coordinates are not supposed to look anything like a natural image, even if you keep them all (number of components = number of pixels). In particular, all the different elements ("pixels") are uncorrelated.

When compressing an image this way, you are saying that each image in your dataset has many pixels, but that the images are different from each other in only 100 ways. You find 100 basis images that represent a typical image well, and then each reconstructed image is a linear combination of these 100 basis images, and the 100 numbers in the compressed vector are the coefficients multiplying each basis image. In order to understand what each "pixel" in the compressed vector means you must plot these basis images. To do this in python, try reshaping the rows of pca.components_

When dealing with face images, these basis images are sometimes called eigenfaces.

If you want to see your images projected on the low-dimensional principal subspace, then after applying:

compressed = pca.transform(faces)

you need to apply

decompressed = pca.inverse_transform(compressed)

and plot the decompressed images.

Improve this answer
$\endgroup$
6
  • $\begingroup$ This doesn't quite answer my question. There must be some correlation between the original images and the reduced vector – does it have to do with areas of contrast or edges like in haar detection? Or is it purely mathematical? $\endgroup$
    – JeffThompson
    Mar 29, 2018 at 10:48
  • 2
    $\begingroup$ @JeffThompson The reduced vector is not supposed to have any resemblance to the original images. The grey levels in the reduced vector are not supposed to look anything like adges or contrast or any grey levels in the original images. The order of the "pixels" in the reduced vectors is in decreasing variance of projections of the images onto smaller dimensional vectors, and the different "pixels" are uncorrelated, so their appearance resembles noise. $\endgroup$
    – elliotp
    Mar 29, 2018 at 11:03
  • $\begingroup$ Got it. Can you tell me in simple or metaphorical terms what they represent? You mention "variance of projections" but I'm not familiar with those terms and am trying to get my head around what has been pulled out of the image :) $\endgroup$
    – JeffThompson
    Mar 29, 2018 at 12:20
  • 1
    $\begingroup$ @JeffThompson When compressing an image using PCA, you are saying that each image in your dataset has many pixels, but the images are different from each other in only 100 ways. You find 100 "basis images" that represent a typical image well. The reconstructed image is a linear combination of 100 "basis images", and the 100 numbers in the compressed vector are the coefficients multiplying each "basis image". You really can't understand what each "pixel" in the compressed vector means without also plotting the "basis images". To do this in python, try reshaping the rows of pca.components_ $\endgroup$
    – elliotp
    Mar 29, 2018 at 12:42
  • $\begingroup$ Thanks, super helpful. If you update your answer with this info, happy to accept it. $\endgroup$
    – JeffThompson
    Mar 29, 2018 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.