How does neural network recognise images?

Question

I am trying to learn how Neural Network works on image recognition. I have seen some examples and become even more confused. In the example of letter recognition of a 20x20 image, the values of each pixel become the input layer. So 400 neurons. Then a hidden layer of neurons and 26 output neurons. Then train the network, and then it works, not perfect.

What confused me about the Neural Network is, how it learns about what's in an image. You don't need to do thresholding,or segmentation, or measurement, somehow the network learns to compare images and recognize. It is like magic to me now. Where to start to learn neural network?

Answer 1

A major insight into how a neural network can learn to classify something as complex as image data given just examples and correct answers came to me while studying the work of Professor Kunihiko Fukushima on the neocognitrion in the 1980's. Instead of just showing his network a bunch of images, and using back-propagation to let it figure things on it's own, he took a different approach and trained his network layer by layer, and even node by node. He analyzed the performance and operation of each individual node of the network and intentionally modified those parts to make them respond in intended ways.

For instance, he knew he wanted the network to be able to recognize lines, so he trained specific layers and nodes to recognize three pixel horizontal lines, 3 pixel vertical lines and specific variations of diagonal lines at all angles. By doing this, he knew exactly which parts of the network could be counted on to fire when the desired patterns existed. Then, since each layer is highly connected, the entire neocognitron as a whole could identify each of the composite parts present in the image no matter where they physically existed. So when a specific line segment existed somewhere in the image, there would always be a specific node that would fire.

Keeping this picture ever present, consider linear regression which is simply finding a formula ( or a line) via sum of squared error, that passes most closely through your data, that's easy enough to understand. To find curved "lines" we can do the same sum of products calculation, except now we add a few parameters of x^2 or x^3 or even higher order polynomials. Now you have a logistic regression classifier. This classifier can find relationships that are not linear in nature. In fact logistic regression can express relationships that are arbitrarily complex, but you still need to manually choose the correct number of power features to do a good job at predicting the data.

One way to think of the neural network is to consider the last layer as a logistic regression classifier, and then the hidden layers can be thought of as automatic "feature selectors". This eliminates the work of manually choosing the correct number of, and power of, the input features. Thus, the NN becomes an automatic power feature selector and can find any linear or non-linear relationship or serve as a classifier of arbitrarily complex sets** (this, assumes only, that there are enough hidden layers and connections to represent the complexity of the model it needs to learn). In the end, a well functioning NN is expected to learn not just "the relationship" between the input and outputs, but instead we strive for an abstraction or a model that generalizes well.

As a rule of thumb, the neural network can not learn anything a reasonably intelligent human could not theoretically learn given enough time from the same data, however,

• it may be able to learn somethings no one has figured out yet
• for large problems a bank of computers processing neural networks can find really good solutions much faster than a team of people (at a much lower cost)
• once trained NNs will produce consitsent results with the inputs they've been trained on and should generalize well if tweaked properly
• NN's never get bored or distracted

Answer 2

You may have heard it said that neural networks are "universal function approximators". In essence, the Cybenko theorem says that for any function mapping reals to reals, you can approximate it with a neural network with sigmoid activation functions. In fact, it turns out that neural networks allow you to compute any function which is computable by a Turing machine (ie anything you can write an algorithm to compute). Unfortunately, these proofs only say that for some finite configuration of neurons and weights, you can approximate any function.

The theory is all nice and dandy, but your question seems to be more along the lines of how to actually encode the computation of some function into a set of neurons and weights. To illustrate, consider a simple example - the exclusive-or. The XOR takes two inputs, passes those inputs. When one and only one of the inputs are activated, then the output node is activated. With both or none of the inputs are activated, then the output node is not activated.

Notice that the three hidden nodes do different things. The left most and right most nodes simply pass through the respect input nodes activations. The middle neuron takes the two inputs and somehow negates them if they are both on. This clever combining and recombining of inputs is essentially how work in a neural network is done.

Obviously for more complex functions the combining and recombining must be done in more clever and complicated ways, but this is in essence what happens at a low level. The crazy thing is that this is really all you need to compute any computable function! Then again, turing machines also turn out to be deceptively simple...

The problem is that we don't really have a way to magically generate the neural network which computes some arbitrary function. The proofs only tell us that there is some network out there that could do it. When we train our neural networks, we are simply trying to find a network which is pretty close.

In the context of image recognition, you could imagine encoding patterns into the network. For example, to recognize the number '1', you could imagine a hidden nodes which expect a column of pixels to be mostly or all activated, with neighboring pixels to be off. This hidden node could be fairly good at recognizing a straight line in that particular column. Put enough of these together and pretty soon you've got a bunch of nodes that do it in enough places of your image that if I show the network a one, enough straight line hidden nodes will be activated, indicating a '1'. The problem of course become generalizing the network so it can recognize a varied set of inputs.

Hopefully this helps you understand more or less the concepts of how a neural network can perform computations. However, you've hit upon a point which is rather important about neural networks: in general it is difficult at best to understand why the network spit out a particular output, especially when you consider that for something like image recognition, the networks are generally big enough that humans have a tough time comprehending each of the moving parts of the machine. Further complicating the matter is that in general most neural networks do not actually have a single hidden node for each little feature the network could learn about the data. Instead, detecting something like a straight line to classify the number '1' would take place in a non-centralized manner over many hidden nodes. Other algorithms, such as decision trees, are much nicer in this respect.

If you are looking for more reading, I highly recommend reading through this tutorial over at ai junkie. It walks you through the basics of how a neural network works, and even gives a simple code example getting neural networks to drive a tank towards a goal. The tutorial does not however cover backpropagation, which is by far the most common way of training neural networks, and instead uses a simple genetic algorithm. Once he starts talking genetics, I guess you can stop reading...

Answer 3

That what confused you is

how it learns about what's in an image.

What is in an image is digitally represented by the values in the image's pixels. If you take an example of color in the image. The pixel may have three values, each for the three main colors - Red, Green and Blue (RGB). A pixel with (10,50,100) means it has 'less' blue color elements than a pixel with (40,50,100). Thus, in the image the first pixel represent a region with less color blue. This is the information the neural network learns, from one location/region of the image to the other and ends up 'knowing' what is in the image. The same principle is applied for other image features (besides color) that may be used as input to neural network. See this,and this for basic image concepts and then move to this to learn how neural network work.

Answer 4

All the machine learning problems are same. You have some train data, learn a model that represent this data and have ability to generalize this knowledge in that way you cluster,classify, learn with different algorithms.

In Image recognition you have again a set of images you want to to learn about.

1. These images firstly processed and some features are extracted from images (lots of possible image feature schemes like SIFT, Bag of WORDS) like you use pixels and their values.
2. Give these images with corresponding feature vectors to your ML algorithm (Neural Net, SVM or others).
3. Learn a model
4. Use this model to recognize objects that are seen sufficiently on training data.

If you want to recognize more than one thing, use multiple classifier for each.

Answer 5

I would also like to mention very popular for image recognition convolutional neural networks. Here is a link to simplified explanation of a CNN.

Briefly, in CNN image is first split into features, like edges, shapes, collections of shapes. Then these features are 'fed' into a 'regular' fully-connected multi-layer neural network (multi-layer perceptron).

In more details, a set of filters are applied to extract features in a form of a feature map. A filter is just a matrix (random in the beginning) that is applied to the original image so that dot product of the original image matrix and filter matrix is calculated and the result is summed up. Filter moves along the original image one pixel (the step) at a time and the matrix of feature map is being filled. A feature map is created for each filter. Then non-linearity introduced with RELU(Rectified Linear Unit) for each pixel in each feature map. Pooling, through application of either max(), sum() or average(), is done after convolution. Finally, features extracted this way look like 'magnified' pieces of the original image. These features are input into a fully connected (all units are connected) neural network and the probabilities for each image (lets say we trained our network on images of cars, trees and boats) are calculated after each feedforward pass through the network. The network is trained, which means that the parameters(weights) and filter matrices are optimized through backpropagation (minimization of misclassification error). When a new image is input into the trained network only feedforward is needed to identify the image (provided that the network is accurate enough, i.e. we trained it with enough examples etc.)

Answer 6

It's good to know ANN can create any function f(x) or f(x,y,z,..) or any multifunction for that matter. But it's also important to know that functions have limits in how they can classify data...there are more complex relations subsets of powersets of objects, which are important in classification and these are not described by such functions in an elegant or natural way, but are important in language and classifying objects. ANN can accomplish this however as well.