0
$\begingroup$

I am taking the Machine Learning course by Andrew Ng on coursera. I am trying to make a neural network learn to do XOR, but I am facing a problem regarding the shapes of the $\delta$ vectors, and $\Delta$ required to implement the Backpropagation algorithm.

Basically the XOR Neural network would have three layers, with 2,2,1 units respectively(not including the bias). Hence $s_{1} = 2$, $s_{2} = 2$,$s_{3} = 1$. And dimension/shapes of the weight matrix theta would be $\theta^{1} = (2,3)$, and $\theta^{2} = (1,3)$. Where dimension of $\theta^{(l)} = (s_{l+1},s_{l}+1)$.

Here is a link to the Resources page on Coursera for referring to the mentioned formulas: https://www.coursera.org/learn/machine-learning/resources/EcbzQ

Now the dimension of $\delta^{3} = (1,)$ in this case,as $\delta^{3}$ is a vector. Now $\delta^{2}$ can be computed using the formula, $\delta^{(l)} = ((\Theta^{(l)})^T \delta^{(l+1)})\ .*\ a^{(l)}\ .*\ (1 - a^{(l)})$. Here $\ .*$ represents element wise multiplication.

But I am facing a problem computing the dimension $\delta^{2}$ using the above. Here is a python code example:

     theta_2 = np.ones((1,3))
     delta_3 = np.ones((1))
     a_2 = np.ones((2))
     delta_2 = np.dot(theta_2.T,delta_3)*a_2*(1-a_2)

Error:

    ValueError  Traceback (most recent call last)

    <ipython-input-28-a2001b7983a3> in <module>()
     2 delta_3 = np.ones((1))
     3 a_2 = np.ones((2))
     ----> 4 delta_2 = np.dot(theta_2.T,delta_3)*a_2*(1-a_2)

   ValueError: operands could not be broadcast together with shapes (3,) (2,) 

If somebody could help me out to complete the Neural Network to learn XOR function. Thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.