I am taking the Machine Learning course by Andrew Ng on coursera. I am trying to make a neural network learn to do XOR, but I am facing a problem regarding the shapes of the $\delta$ vectors, and $\Delta$ required to implement the Backpropagation algorithm.

Basically the XOR Neural network would have three layers, with 2,2,1 units respectively(not including the bias). Hence $s_{1} = 2$, $s_{2} = 2$,$s_{3} = 1$. And dimension/shapes of the weight matrix theta would be $\theta^{1} = (2,3)$, and $\theta^{2} = (1,3)$. Where dimension of $\theta^{(l)} = (s_{l+1},s_{l}+1)$.

Here is a link to the Resources page on Coursera for referring to the mentioned formulas: https://www.coursera.org/learn/machine-learning/resources/EcbzQ

Now the dimension of $\delta^{3} = (1,)$ in this case,as $\delta^{3}$ is a vector. Now $\delta^{2}$ can be computed using the formula, $\delta^{(l)} = ((\Theta^{(l)})^T \delta^{(l+1)})\ .*\ a^{(l)}\ .*\ (1 - a^{(l)})$. Here $\ .*$ represents element wise multiplication.

But I am facing a problem computing the dimension $\delta^{2}$ using the above. Here is a python code example:

     theta_2 = np.ones((1,3))
     delta_3 = np.ones((1))
     a_2 = np.ones((2))
     delta_2 = np.dot(theta_2.T,delta_3)*a_2*(1-a_2)


    ValueError  Traceback (most recent call last)

    <ipython-input-28-a2001b7983a3> in <module>()
     2 delta_3 = np.ones((1))
     3 a_2 = np.ones((2))
     ----> 4 delta_2 = np.dot(theta_2.T,delta_3)*a_2*(1-a_2)

   ValueError: operands could not be broadcast together with shapes (3,) (2,) 

If somebody could help me out to complete the Neural Network to learn XOR function. Thanks in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.