Is it possible to train neural networks such that certain activations are rewarded and some other activations are penalized? In other words, I would like the network to generate preferred values more often. An examples of this would be:
Assume that a neuron has 25 inputs and the values of inputs are restricted to [-1, 1] interval. I have quantized the inputs so that each input has 3 possible values. As a result, there are 3^25 potential combinations of the inputs. I would like the majority of these combinations to produce an output that is greater than 0.5. Additionally, the smaller portion of combinations produce values in the [-1, 0.5) range.
Can I do this by somehow modifying the activation function? If yes, how should I enforce the requirements?
In general, would a network that is trained this way be able to perform classification as well as a network that does not have such limitations?