# Understand neural network in a 'mathematical' way

This is a soft question. But as I read papers/reports about neural network used for pictures. Often there are comments like 'the first layer of the neural network captures the edge/shape information' etc.

I am just wondering what exactly the 'edge/shape' is if we just view an image as a matrix. Meanwhile, is there any intuition that a neural network could capture the 'edge/shape' if we just view a neural network as a composition of linear and non-linear functions?

Thanks so much!

• As a heads up, the particular type of neural network to which you are referring is a convolutional neural network---really clever idea. Brandon Rohrer has an awesome YouTube video about the topic. After you watch it, I suggest drawing out what a 2x2 filter does to a 3x3 image. I never really got why a CNN is a neural network until I did that. It will also help you see how clever a CNN is in putting restrictions on weights because we know something about the input data and don't want to weights fluctuating willy nilly.
– Dave
Aug 30, 2019 at 1:15

I am just wondering what exactly the 'edge/shape' is if we just view an image as a matrix.

Basically, neural networks are learning so-called filters, which is a concept from traditional computer vision. In traditional computer vision, these were developed manually. For instance, if you have a convolutional neural network with a 3x3 kernel that has the following weights:

$$k = \begin{bmatrix} -1 & -2 & -1 \\ 0 & 0 & 0\\ 1 & 2& 1 \end{bmatrix}$$

then, you have a so-called horizontal Sobel filter that can detect vertical edges if you use it for convolution (sliding it over the input image). For instance, see the illustration below:

(Image source: https://i.stack.imgur.com/RBFN4.png)

Let's take a practical example using the vertical Sobel (vertical edge detector):

import torch
import numpy as np
from torchvision import datasets
from torchvision import transforms

conv = torch.nn.Conv2d(1, 1, kernel_size=3)
conv.weight = torch.nn.Parameter(torch.tensor([[[[-1., 0., 1.],
[-2., 0., 2.],
[-1., 0., 1.]]]]))



Let's load an input image first:


train_dataset = datasets.MNIST(root='data',
train=True,
transform=transforms.ToTensor(),

batch_size=10,
shuffle=False)

break

import matplotlib.pyplot as plt

nhwc_img = np.transpose(images[7], axes=(1, 2, 0))
nhw_img = np.squeeze(nhwc_img.numpy(), axis=2)
plt.imshow(nhw_img, cmap='Greys');



And now apply the vertical Sobel filter we defined above:

with torch.set_grad_enabled(False):
nhwc_img = conv(images[7].view(1, 1, 28, 28))
plt.imshow(torch.squeeze(nhwc_img), cmap='Greys')


As you can see, the signal in the output is primarily there where the input had vertical edges. Of course, the network will probably not learn Sobel filters but learn to detect filters in a way that are useful for minimizing a loss function on the given task. Hopefully, this provides some intuition though.