Questions tagged [backpropagation]

Backpropagation, an abbreviation for "backward propagation of errors", is a common method of training artificial neural networks used in conjunction with an optimization method such as gradient descent.

356 questions
Filter by
Sorted by
Tagged with
23 views

Suspicion about the errorback propagation formula in PRML

From PRML: Do variations in $a_j$ give rise to variations in $E$ only via $a_k$? True. Is $h'(a_j)$ the same across all $k$? I'm afraid not. Let's write down what $\partial a_k/\partial a_j$ are....
387 views

Why can't backpropagation be used for binary threshold neurons

I was reading that backpropagation can't be used for binary threshold neurons. Binary threshold neurons calculate a cost using some weights, so why can't those weights be changed with backpropagation?...
332 views

When doing backpropagation, why do we multiply with the derivative of the activation function

I was reading about backpropagation, which in my case works like this: I don't understand why when computing the error term, we are multiplying with the derivative of the activation function: Why ...
402 views

Batch-normalization back propagation equation

I have been trying to understand applying the back propagation on batch-normalization. However, the matrix multiplication and equations involved have got me confused and I feel that my understanding ...
848 views

2k views

resilient backpropagation parameters selection

In the original paper for resilient backpropagation (http://paginas.fe.up.pt/~ee02162/dissertacao/RPROP%20paper.pdf), the author says "One of the main advantages of RPROP lies in the fact, that for ...
1k views

How to interpret the classification boundary?

I am beginner to neural network and machine learning. I am working neural network with 1 hidden layer. I took spiral data set and I am trying to overfit the data. I applied neural network to it and I ...
97 views

Probability theoretic analog to Neural Networks

I am trying to figure out how to do learning using probabilistic programming languages. For this I am following different paths to get a hold on the way of thinking. I understand modelling using ...
94 views

Derive the gradients of a basic neural network

Given a neural network as following \begin{align*} &J = CE(y,\hat{y})=-\sum_i y_i log(\hat{y}_i)\\ &\hat{y} = softmax(z_2)\\ &z_2 = hW_2+b_2\\ &h = sigmoid(z_1)\\ &...
3k views

I am going through the problems in the Stanford NLP deep learning class's written assignment problems http://cs224d.stanford.edu/assignment1/assignment1_soln I am trying to understand the answer for ...
136 views

Backpropagation - computing partial derivative with respect to W

I am following a chapter on backprop derivation from the online book by Michael Nielsen In particular, following equation is derived in Chapter 2: ${∂C\over∂w^{l}_{jk}}=a^{l−1}_{k}δ^{l}_{j}$ Now, I ...
33 views

does bp need to be simultanious? - XOR learning problem

I have a problem with XOR learning. I have 2(inputs),2,1(output) neuron neural network with sigmoidal function and normal sets ...
126 views

Matrix-based implementation for an ANN with different number of nodes per hidden layer?

Given a neural network architecture that has multiple hidden layers and a different number of nodes in each hidden layer (assuming that this is a valid architecture), is it possible to implement feed ...
70 views

108 views

Backpropagation not incremental

Why is backpropagation not incremental? The definition of incremental would be to be able to update the weights with every single new data point. However, in stochastic gradient descent with a ...
379 views

Backprop: Calc gradients for embed layers and how to do gradient check

First I want to check is I understand correctly how backprop works for NN with embed layers. Lets create simplest NN with embed layer Input Layer [i1; i2; i3; i4] - input nodes Embed layer [h1] [h2] ...