# Normal Neural Network for Image 100×100? [duplicate]

So, I have created a neural network using back propagation and sigmoid function. It seems to work for XOR and images with size of 28×28. However, When I input it 100×100 image the mean-square-error is 0.3 ish.

I am using 1 hidden layer. I basically have two questions:

1. Is it possible to use neural network not deep neural network to learn of an image the size of 100×100? If so, could you give me a detailed explanation, I have posted the code that I wrote below, sorry in advance as it is not the best or the cleanest code.
2. When passing error gradient in a deep neural network for 2 hidden layers do you pass the hidden gradient calculated by the output layer to the 2nd hidden layer and this 2nd hidden layer then calculates the gradient descent/hidden error gradient for the 1st hidden layer is this correct? Whilst updating the weights.

• A1: It is possible for some classification problems. Having a single hidden layer limits the complexity of function that the NN can represent. The number of nodes in the hidden layer will affect the complexity too. The outcome will also depend on the number of classes. I didn't look at your code in detail, but it seems you have 1 output node. Usually the output layer has as many nodes, as number of classes in the data set. I don't have a formal way to determine what is a good size for the hidden layer, but I'd try with 150. A2: Yes, you are correct - that's why it's back propagation. Sep 5, 2016 at 19:00
• If I try it with 150 nodes in hidden layer it outputs 0.2 as mse and gets stuck in it. Thank you for clearing Q2 Sep 5, 2016 at 19:12
• Did you also bump the number of nodes in the output layer to the number of classes? Sep 5, 2016 at 19:15
• Nope do I bump it to 2 Sep 5, 2016 at 19:25
• As I have increased the nodes in the output it seems to have better results Sep 5, 2016 at 19:34

First question: Yes, why should it not be possible ?

Second question: I would propose to tackle the backpropagation topic first on the paper by understanding why the weight-update-rule looks the way it does. For this you need only partial derivation as mathematical tools: deriving the error function with respect to the net weights. You will see that for hidden layers that error is actually not given directly (because there is no present expected output as for the output layer). The backpropagation algorithm is simply way of propagating that output error backwards and give the hidden-layers the possibility of knowing how much their weights are influencing the decision made at the output layer in order to update them accordingly.

Generally the architecture of classical feedforward networks depends heavily on the problem that needs to be solved (furthermore problems are devided into two types, namely regression and classification problems).

So lets say your task is to classify images of handwritten images provided as 28x28 matrices of gray-scale values (like the famous MNIST dataset) there is no strict specification on how much hidden layers your net has to have or how the output layer should look like.

As was already stated here: The output layer for some digit-classification problem may have 10 neurons in the output-layer (whereby neuron with highest output would point to the currently made net-decision). But there is no rule that this is the only way to go.

The answer is stat in the above comment:

A1: It is possible for some classification problems. Having a single hidden layer limits the complexity of function that the NN can represent. The number of nodes in the hidden layer will affect the complexity too. The outcome will also depend on the number of classes. I didn't look at your code in detail, but it seems you have 1 output node. Usually the output layer has as many nodes, as number of classes in the data set. I don't have a formal way to determine what is a good size for the hidden layer, but I'd try with 150.

A2: Yes, you are correct - that's why it's back propagation

this seems to work for me. Thanks @iliyan bobev

• If you believe this is the answer to your question, please consider accepting it by clicking the check mark below the vote total. Mar 15, 2017 at 15:52