I've recently seen tutorials from google's tensor flow.

It was about convolutional network and way of convolution way slightly different from what I first learned.


The way I first learned about convolution was Coates et al. approach (http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2011_CoatesNL11.pdf)

What I know as convolution was do k-means to create K centroids of image patches and use it as convolution network neurons.

So in the training process, single image patch is transformed to K features (distance to centroids). I understaned this as centroids being some kind of neurons and distance to centroids are activation.

However, in the tensorflow example convolution layer seems like just typical neural net with ReLU activation function but what is does is applying same neurons to different patches. There seems no k-means nor unsupervised learning techniques to make features.

I'm not sure what I said about tensorflow example is correct. Is tensorflow example's convolution layer is just same neuron(from MLP)'s network applied to different image patches and optimizing weights of these networks with gradient descent? Without k-means or other unsupervised things?

What is real convolution network anyway between these two?

If both is convolution network and convolution is just a way of applying same neurons, which of these two is more dominant and considered 'better' and 'standard' way of implementing CNN?

Thank you for reading and have a good day


The tensorflow example is what a convolutional neural network generally refers to.

Though I found Coates's paper very interesting and profound, I think the term "Convolutional extraction" used might seem somewhat misleading.

If I understood correctly, the way it extracts patches is the same as CNNs, but it uses a set of fixed features learned unsupervisedly instead of performing 2D convolutions with adjustable weights, which makes it different from the common definition of CNNs.

| cite | improve this answer | |
  • $\begingroup$ @amityaffliction you're welcome $\endgroup$ – dontloo Jan 23 '16 at 2:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.