While building a neural network with one hidden layer, the question arose whether or not to update the biases during backpropagation. I'm basically trying to save up on memory, so my question was and is how big a difference it would make if I updated only the weights compared to the weights and the biases. With the former, I wouldn't have to save any other bias value than the value of `1` which I set as standard. Will it have trouble learning then? If so, then why are the weights updates insufficient for training it?