Basically I am trying to create a Neural Network in c# from scratch, without using any libraries. The issue that I am facing with right now is whenever I try to train my network with different inputs it simply does not work.
For example, I tried solving a basic problem like the XOR logic gate with my NN. Whenever i use the complete training set (all the possible combinations of 1s and 0s) my NN cannot set the weights in such a way that it could produce the desired output. If I train my NN with all the possible variation of inputs after a while instead of setting the weights properly it sets them in a way that the output in each case is around 0.5. On the other hand, if I train it with only one input combination (e.g.: 1,1) it gives the desired output (0).
Any idea why it could be? I use backpropagation to change the weights. My activation function is sigmoid and the cost function is the squared error function. NN has one hidden layer. I also observed that the actual value changes depending on the type of problem. For example when I tried the NN on OR gate it was ~0.75. (Not surprising given that it will be "on" 3 times out of 4)