Implementing Neural Net - Weights Matrix

Question

I'm trying to implement a simple neural network to help me understand the concept. What I'm now not sure about is how the matrix of weights is formatted.

Example:

• Input layer: 2 inputs
• Hidden layer: 2 nodes
• Output layer: 1 node

From what I've understood, this is what I send as input to the output layer:

outputFromLayerOne = [
  sigmoid (
    (
     (input[0] * weights[0] + bias) +
     (input [1] * weights[1] + bias)
    )
  ), sigmoid (
    (
     (input[0] * weights[0] + bias) +
     (input [1] * weights[1] + bias)
    )
  )]

In other words, since there are 2 nodes in the hidden layer 1, there is are only two weights that get applied against the two inputs. Is this how it is set up, or are there two weights for each node in my hidden layer?

Answer 1

If the following diagram represents the architecture of the NN you describe,

---

---

then the weight matrix of the hidden layer would be:

$$\begin{bmatrix} \color{blue}{\theta_{1,0}^{(1)}} && \color{red}{\theta_{2,0}^{(1)}}\\ \color{blue}{\theta_{1,1}^{(1)}} && \color{red}{\theta_{2,1}^{(1)}}\\ \color{blue}{\theta_{1,2}^{(1)}} && \color{red}{\theta_{2,2}^{(1)}} \end{bmatrix}$$

And for the output layer:

$$\begin{bmatrix}\color{brown}{ \theta_{1,0}^{(2)}\\ \theta_{1,1}^{(2)}\\ \theta_{1,2}^{(2)}} \end{bmatrix}$$