I am learning a machine learning class online from Stanford, namely CS 229. There is one section about deep learning and back-propagation in deep learning.

  The network looks like:

The forward propagation can be defined as:

where g is the activation function.

The dimensions of each variable can also be given as:

Now, for back-propagation, by using chain rule, we can get:

To match up with the dimensions, we have:

I know that after applying chain rule, the normal way is to calculate generalized Jacobian matrix and do matrix multiplication. However, the dimension of each part in chain rule above does not match what generalized Jacobian matrix will give us. For example, for the last term in chain rule, the dimension from generalized Jacobian matrix should be (2 X 1) X (2 X 3). However, what course notes say is 1 X 3.

Why is it true?

Any comments are appreciated!

I have a question about the dimensions of a Jacobian during backpropagation. The network looks like:

The forward propagation can be defined as:

where g is the activation function.

The dimensions of each variable can also be given as:

Now, for back-propagation, by using chain rule, we can get:

To match up with the dimensions, we have:

I know that after applying chain rule, the normal way is to calculate generalized Jacobian matrix and do matrix multiplication. However, the dimension of each part in chain rule above does not match what generalized Jacobian matrix will give us. For example, for the last term in chain rule, the dimension from generalized Jacobian matrix should be (2 X 1) X (2 X 3). However, what course notes say is 1 X 3.

Why is it true?

Any comments are appreciated!

I am learning a machine learning class online from Stanford, namely CS 229. There is one section about deep learning and back-propagation in deep learning.

The network looks like:

The forward propagation can be defined as:

where g is the activation function.

The dimensions of each variable can also be given as:

Now, for back-propagation, by using chain rule, we can get:

To match up with the dimensions, we have:

I know that after applying chain rule, the normal way is to calculate generalized Jacobian matrix and do matrix multiplication. However, the dimension of each part in chain rule above does not match what generalized Jacobian matrix will give us. For example, for the last term in chain rule, the dimension from generalized Jacobian matrix should be (2 X 1) X (2 X 3). However, what course notes say is 2 X 3.

Why is it true?

Any comments are appreciated!

I am learning a machine learning class online from Stanford, namely CS 229. There is one section about deep learning and back-propagation in deep learning.

The network looks like:

The forward propagation can be defined as:

where g is the activation function.

The dimensions of each variable can also be given as:

Now, for back-propagation, by using chain rule, we can get:

To match up with the dimensions, we have:

I know that after applying chain rule, the normal way is to calculate generalized Jacobian matrix and do matrix multiplication. However, the dimension of each part in chain rule above does not match what generalized Jacobian matrix will give us. For example, for the last term in chain rule, the dimension from generalized Jacobian matrix should be (2 X 1) X (2 X 3). However, what course notes say is 1 X 3.

Why is it true?

Any comments are appreciated!