4
$\begingroup$

This is a soft question. But as I read papers/reports about neural network used for pictures. Often there are comments like 'the first layer of the neural network captures the edge/shape information' etc.

I am just wondering what exactly the 'edge/shape' is if we just view an image as a matrix. Meanwhile, is there any intuition that a neural network could capture the 'edge/shape' if we just view a neural network as a composition of linear and non-linear functions?

Thanks so much!

$\endgroup$
1
  • $\begingroup$ As a heads up, the particular type of neural network to which you are referring is a convolutional neural network---really clever idea. Brandon Rohrer has an awesome YouTube video about the topic. After you watch it, I suggest drawing out what a 2x2 filter does to a 3x3 image. I never really got why a CNN is a neural network until I did that. It will also help you see how clever a CNN is in putting restrictions on weights because we know something about the input data and don't want to weights fluctuating willy nilly. $\endgroup$
    – Dave
    Aug 30, 2019 at 1:15

1 Answer 1

5
$\begingroup$

I am just wondering what exactly the 'edge/shape' is if we just view an image as a matrix.

Basically, neural networks are learning so-called filters, which is a concept from traditional computer vision. In traditional computer vision, these were developed manually. For instance, if you have a convolutional neural network with a 3x3 kernel that has the following weights:

$$ k = \begin{bmatrix} -1 & -2 & -1 \\ 0 & 0 & 0\\ 1 & 2& 1 \end{bmatrix} $$

then, you have a so-called horizontal Sobel filter that can detect vertical edges if you use it for convolution (sliding it over the input image). For instance, see the illustration below:

enter image description here

(Image source: https://i.stack.imgur.com/RBFN4.png)

Let's take a practical example using the vertical Sobel (vertical edge detector):

import torch
import numpy as np
from torchvision import datasets
from torchvision import transforms
from torch.utils.data import DataLoader


conv = torch.nn.Conv2d(1, 1, kernel_size=3)
conv.weight = torch.nn.Parameter(torch.tensor([[[[-1., 0., 1.],
                                                 [-2., 0., 2.],
                                                 [-1., 0., 1.]]]]))

Let's load an input image first:


train_dataset = datasets.MNIST(root='data', 
                               train=True, 
                               transform=transforms.ToTensor(),
                               download=True)

train_loader = DataLoader(dataset=train_dataset, 
                          batch_size=10, 
                          shuffle=False)

for images, labels in train_loader:
    break


import matplotlib.pyplot as plt

nhwc_img = np.transpose(images[7], axes=(1, 2, 0))
nhw_img = np.squeeze(nhwc_img.numpy(), axis=2)
plt.imshow(nhw_img, cmap='Greys');

enter image description here

And now apply the vertical Sobel filter we defined above:

with torch.set_grad_enabled(False):
    nhwc_img = conv(images[7].view(1, 1, 28, 28))
    plt.imshow(torch.squeeze(nhwc_img), cmap='Greys')

enter image description here

As you can see, the signal in the output is primarily there where the input had vertical edges. Of course, the network will probably not learn Sobel filters but learn to detect filters in a way that are useful for minimizing a loss function on the given task. Hopefully, this provides some intuition though.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.