2
$\begingroup$

I've read these following papers:

  • Wagner et al. (2013). Learning Convolutional Neural Networks From Few Samples.
  • Ren et al. (2016). Convolutional Neural Network Based on Principal Component Analysis Initialization for Image Classification.

And they mention "PCA to initialize Convolutional Neural Network". I don't really understand, how did they do that?

PCA result is a eigenvector matrix, is it used as kernel in CNN? but does it make the kernel size is too big? or is it used as feature map?

I hope somebody can help explain to me.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

Wagner et al. describe what they do in section 5c. They perform PCA on 7x7 pixel image patches (not whole images), treating patches as points and pixels as dimensions. This gives 49 principal components, each with a 49 element weight vector (which is an eigenvector of the covariance matrix). Reshaping each PCA weight vector to a 7x7 matrix gives a basis image patch (i.e. each original image patch can be expressed as a linear combination of the basis patches). Each basis patch is used as an initial filter kernel in the convolutional network, yielding 49 filters.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Oh I see, so each eigenvector become a kernel (it results 49 kernels) with 7x7 size, isn't it? $\endgroup$
    – malioboro
    Commented Aug 31, 2017 at 2:07
  • $\begingroup$ Yes, that's right $\endgroup$
    – user20160
    Commented Aug 31, 2017 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.