I am looking at an example for the activation function $a_1$.

Why does the equation look like $\Theta_{10}x_0 + \Theta_{11}x_1 + \Theta_{12}x_2 + \Theta_{13}x_3$

instead of

$\Theta_{10}x_0 + \Theta_{11}x_1 + \Theta_{21}x_2 + \Theta_{31}x_3$?

My understanding is that all input nodes will contribute to the computation of activation value of $a_1$. And hence, shouldn't $\Theta_{xy}$ be the value from $x$ to $y$? Which in this case of $\Theta_{21}$ be from Node $2$ in first layer to Node $1$ in second layer. Am I misunderstanding something?

This is the url http://www.holehouse.org/mlclass/08_Neural_Networks_Representation_files/Image%20[5].png


To quote the very page the image is from:


  • $j$ (first of two subscript numbers)= ranges from $1$ to the number of units in layer $l+1$
  • $i$ (second of two subscript numbers) = ranges from $0$ to the number of units in layer $l$
  • $l$ is the layer you're moving FROM

I guess the choice for this is to be able to write it with matrices:

$ \underbrace{ \begin{pmatrix} \Theta_{00} & \Theta_{01} & \Theta_{02} & \Theta_{03} \\ \Theta_{10} & \Theta_{11} & \Theta_{12} & \Theta_{13} \\ \Theta_{20} & \Theta_{21} & \Theta_{22} & \Theta_{23} \\ \end{pmatrix} }_{\Huge\Theta_{ji}} * \begin{pmatrix} x_{0} \\ x_{1} \\ x_{2} \\ x_{3} \end{pmatrix} = \begin{pmatrix} \Theta_{00}x_0 & \Theta_{01}x_1 & \Theta_{02}x_2 & \Theta_{03}x_3 \\ \Theta_{10}x_0 & \Theta_{11}x_1 & \Theta_{12}x_2 & \Theta_{13}x_3 \\ \Theta_{20}x_0 & \Theta_{21}x_1 & \Theta_{22}x_2 & \Theta_{23}x_3 \\ \end{pmatrix} \begin{matrix} (\text{input }a_1) \\ (\text{input }a_2) \\ (\text{input }a_3) \\ \end{matrix} $

$x_0$ is bias by the way.

With $\Theta_{ij}$ you'd have $x^T\Theta_{ij}$ where every column $c$ would be input to $a_c$ but its custom to have the vector after the matrix.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.