I am trying to program an OCR based on neural networks.
It has in sum 3 layers : Input( 32 * 32 ) => Hidden (180) => Output(2)
It task is to recognize the numbers 1 and 2. If I let the network train for example to figure out what "1" is, I set as target [1, 0]. It backpropagates until the output is less than the treshhold( id est the output of the desired neurone is 0.999 then the treshold is calculated as (1 - output)). If I train now the number 2 ( target [0, 1]) it trains correctly to [1E-7, 0.9] but forgets the old training and therefore tends in the output layer more likely to the number "2".
How can I train the neural network to the effect, that both trainings still effecting the output? Because the training for number "2" is newer, it is more weighted in the output. If I give as input an "1" the output will something like [0.1, 0.2]...
To better understand: Given is a training set of ten examples of number "1" and ten examples of number "2". I first let the network_ train the number 1 with these ten examples until the output ist something like [0.999, 0.001]. The problem is now that whenever I try to train the network for the number 2, the error is extremly big, because the network is trained to recognize the number "1". How can I _let the network train for recognizing "1" AND "2" without losing the information of the last training?
Thanks in advance!