# Questions tagged [backpropagation]

Backpropagation, an abbreviation for "backward propagation of errors", is a common method of training artificial neural networks used in conjunction with an optimization method such as gradient descent.

357 questions
Filter by
Sorted by
Tagged with
2k views

733 views

### Backpropagation for Bias in Neural Networks

I have a problem in my neural network relating to the bias vector. I'm using this source as a reference. My understanding of calculating the bias is that the partial derivative cost function with ...
92 views

### Under periodic BPTT, is softmax evaluated only at the end of the period?

Suppose I have a continuous sequence $X$ of words and I wish to train a RNN language model. According to , I would split $X$ into subsequences $X^{1..|X|/k_1}$ $k_1$ sized subsequences ($k_1$ is ...
34 views

### Neural Network probabilities converging to biases

I'm creating an Android app which can use a variety of classification formula, and while I have normal Softmax done correctly, I keep having an issue with the Softmax Neural Network. After about 10 ...
80 views

### how does a neural network with stochastic backpropagation make sure it doesn't “undo” previous learning?

Assume we have a neural network with stochastic gradient descent used for backpropagation, and therefore each element in the training set is used once to calculate the error, and then to adjust the ...
197 views

### Backpropagation Through Time Error Computation

I'm attempting to work through the backpropagation through time terms using this source: http://www.deeplearningbook.org/contents/rnn.html The final formulas are given on pages 385 and 386, but I ...
856 views

### Is it normal that a Neural Network sometimes doesn't learn Xor?

I've implemented a neural network and I'm training it to compute Xor. 1 out of x times it fails to learn, where x is about 5 or 10. It then gives e.g. 0.67 instead of 0 as output for input (1,1). Is ...
146 views

### Neural Networks — How to design for multiple outputs [duplicate]

Given a multilayer perceptron to be used to separate 2 classes. One has 2 design options. 1 -- Use one output node -- where one class is trained to give an output equal to zero, and the other class ...
2k views

### Simple Neural Network for time series prediction

I am creating a simple Multi-layered feed forward Neural Network using AForge.net NN library. My NN is a 3 Layered Activation Network trained with Supervised Learning approach using BackPropogation ...
385 views

### Where can I find a clear derivation of backpropagation through a Convolutional Neural Network?

Any links to books, articles or papers would be appreciated, or even a written explanation.
205 views

### Having trouble understanding/implementing backpropagation algorithm

I have a simple feedforward neural network with 2 input neurons (and 1 bias neuron), 4 hidden neurons (and 1 bias neuron), and one output neuron. The feedforward mechanism seems to be working fine, ...
306 views

### What are the techniques used for learning in non-feedforward neural networks?

Suppose our network architecture has a hidden layer in which the hidden units are interconnected, then is there some sort of variation on backpropagation that is used? What about in general recurrent ...