Can a convolutional NN be made with perceptrons?

Question

I was reading this interesting article on convolutional neural networks. It showed this image, explaining that for every receptive field of 5x5 pixels/neurons, a value for a hidden value is calculated.

We can think of max-pooling as a way for the network to ask whether a given feature is found anywhere in a region of the image. It then throws away the exact positional information.

So max-pooling is applied.

With multiple convolutional layers, it looks something like this:

But my question is, this whole architecture could be build with perceptrons, right?

For every convolutional layer, one perceptron is needed, with layers:

input_size = 5x5;
hidden_size = 10; e.g.
output_size = 1;

Then for every receptive field in the original image, the 5x5 area is inputted into a perceptron to output the value of a neuron in the hidden layer. So basically doing this for every receptive field:

So the same perceptron is used 24x24 amount of times to construct the hidden layer, because:

is that we're going to use the same weights and bias for each of the 24×24 hidden neurons.

And this works for the hidden layer to the pooling layer as well, input_size = 2x2; output_size = 1;. And in the case of a max-pool layer, it's just a max() function on an array.

and then finally:

The final layer of connections in the network is a fully-connected layer. That is, this layer connects every neuron from the max-pooled layer to every one of the 10 output neurons.

which is a perceptron again.

So my final architecture looks like this:

-> 1 perceptron for every convolutional layer/feature map
-> run this perceptron for every receptive field to create feature map
-> 1 perceptron for every pooling layer
-> run this perceptron for every field in the feature map to create a pooling layer
-> finally input the values of the pooling layer in a regular ALL to ALL perceptron

Or am I overseeing something? Or is this already how they are programmed?

Answer 1

Much depends on how strict you interpret the definition of perceptron. Typically the perceptron refers to the precursor of what is called a neuron in neural networks: it is a linear binary classification function that only allows as output a 1 or a 0.

According to this definition, the answer to your question would be 'no' since convolutional networks require more than a binary output and the max-pooling layer is not linear.

Much confusion comes from the name of the successor to perceptrons: multi-layer perceptrons. MLP's no longer have linear activation functions and are no longer restricted to binary outputs.

MLP's are also different from convolutional networks however in the sense that all layers are fully-connected.

Update: Can MLP's be used to construct a fully convolutional network? You could argue that you could use MLP's with one layer to construct parts of the convolutional network, but by definition MLP's have multiple layers, so again the answer would be 'no'.

Answer 2

Each convolutional layer is essentially (quite like) a perceptron since it's just computing linear sums of the inputs, which are then passed through some non-linearity (like a sigmoid or ReLU). However, note that a defining feature of the convolutional layer is that it applies each linear filter at every position of the image (or at every position of the previous layer). This is not true for a perceptron, which would have to have a separate filter for each position. This also has important consequences for the way the network is trained. In a convolutional net, you have to learn each filter only once. Using perceptrons, you'd have to learn the same filter separately for each position in the image (which is a harder learning problem).

I guess you could say that this means each convolutional filter is simply a set of many identical but shifted perceptrons, but at that point I think you'd be stretching the definitions a bit. So in short, the two are clearly related but (at least in my view) the convolution operation is a defining characteristic that sets the convolutional net apart from perceptrons.