Neural network Equation question

Question

I am looking at an example for the activation function $a_1$.

Why does the equation look like $\Theta_{10}x_0 + \Theta_{11}x_1 + \Theta_{12}x_2 + \Theta_{13}x_3$

instead of

$\Theta_{10}x_0 + \Theta_{11}x_1 + \Theta_{21}x_2 + \Theta_{31}x_3$?

My understanding is that all input nodes will contribute to the computation of activation value of $a_1$. And hence, shouldn't $\Theta_{xy}$ be the value from $x$ to $y$? Which in this case of $\Theta_{21}$ be from Node $2$ in first layer to Node $1$ in second layer. Am I misunderstanding something?

This is the url

Answer 1

To quote the very page the image is from:

$\Theta_{ji}^{(l)}$

• $j$ (first of two subscript numbers)= ranges from $1$ to the number of units in layer $l+1$
• $i$ (second of two subscript numbers) = ranges from $0$ to the number of units in layer $l$
• $l$ is the layer you're moving FROM

I guess the choice for this is to be able to write it with matrices:

$ \underbrace{ \begin{pmatrix} \Theta_{00} & \Theta_{01} & \Theta_{02} & \Theta_{03} \\ \Theta_{10} & \Theta_{11} & \Theta_{12} & \Theta_{13} \\ \Theta_{20} & \Theta_{21} & \Theta_{22} & \Theta_{23} \\ \end{pmatrix} }_{\Huge\Theta_{ji}} * \begin{pmatrix} x_{0} \\ x_{1} \\ x_{2} \\ x_{3} \end{pmatrix} = \begin{pmatrix} \Theta_{00}x_0 & \Theta_{01}x_1 & \Theta_{02}x_2 & \Theta_{03}x_3 \\ \Theta_{10}x_0 & \Theta_{11}x_1 & \Theta_{12}x_2 & \Theta_{13}x_3 \\ \Theta_{20}x_0 & \Theta_{21}x_1 & \Theta_{22}x_2 & \Theta_{23}x_3 \\ \end{pmatrix} \begin{matrix} (\text{input }a_1) \\ (\text{input }a_2) \\ (\text{input }a_3) \\ \end{matrix} $

$x_0$ is bias by the way.

With $\Theta_{ij}$ you'd have $x^T\Theta_{ij}$ where every column $c$ would be input to $a_c$ but its custom to have the vector after the matrix.