How do I calculate output of a Neural Network?

Question

I just started learning about ANNs about a week ago with no classical training. Just by watching videos and reading blogs/white papers, I've gotten this far.

I have a question about the final output of the ANN.

Say for instance I was building a XOR with two input node, 3 nodes in one hidden layer and one node in the output layer. A 2 x 3 x 1.

First I would like to make sure I have the first part right.

So each node has a weight associated with it for each node in the hidden layer, if you have 5 nodes in the hidden layer, the input node would calculate it's input and multiply it by a weight associated with each node in the hidden layer.

So to calculate the sigmoid for the first node, you would take all the inputs and multiply it by the weight (no + for a bias) and apply the sigmoid function for the sum of the inputs * weights. Then we would squash that value with a sigmoid and get 0.5866175789173301.

Essentially, it would be, (1 x .25) + (1 x .10) = .35.

Now, that I just do this three times for each node and get 3 squashed numbers.

  // (input1 * HiddenNode(x)Weight) + (input2 * HiddenNode(x)Weight)
  activationFunction((1 * .25) + (1 * .10)) // 0.5866175789173301
  activationFunction((0 * .40) + (1 * .60)) // 0.6456563062257954
  activationFunction((1 * .20) + (0 * .80)) // 0.549833997312478

Now from what I understand, I again sum & squash those answers:

  activationFunction(hidden1 + hidden2 + hidden3) // 0.8559569515861635

Do I have it correct so far?

My question is, if you're feeding in two scaled numbers to predict grades, 89 & 6.5 = (grade/hours of sleep)

How would you calculate the output from .8559 to a number like 93 and calculate the error on that value? Am I missing anything besides a bias?

If I entered in the percent of change for the last 3 stock price changes, and I wanted it to guess the fourth price, how would I convert an answer like this:

 activationFunction(hidden1 + hidden2 + hidden3) // 0.8559569515861635

to an answer to like .10 (percent change in stock price) or any other real world answer?

Thanks in advance!

Answer 1

Unlike people mentioned. Inputs should not be binary. They should be between a certain range (0,1 for sigmoid, -1,1 for TanH).

On the first part you are exactly right if you don't account for bias.

// Completely right, each hidden node gets input from 2 input nodes
activationFunction((1 * .25) + (1 * .10)) // 0.5866175789173301
activationFunction((0 * .40) + (1 * .60)) // 0.6456563062257954
activationFunction((1 * .20) + (0 * .80)) // 0.549833997312478

// However, all the hidden nodes are connected the output node
output = activationFunction((0.59 * weight1) + (0.64 * weight2) + (0.55 * weight3))

Always keep in mind that nodes can only be connected to other nodes by connections, which always have a weight.

My question is, if you're feeding in two scaled numbers to predict grades, 89 & 6.5 = (grade/hours of sleep)

First you scale the inputs (read more here):

89 > 0.89
6.5 > 6.4 / 24 = 0.27

So if the new grade you got was 100, and your output was 0.8559 then the error on your output node is 1.00 - 0.8559 = 0.1441. Then you backpropagate this through the network, but i'm not the right one to explain that for you.