Suppose I have a multi-layer neural network, with non-linear activation functions, and no bias terms. Is it possible to train this network such that for any input $x$, it will always give the same (or nearly the same) output $y$?
For tanh, relu or linear activations where all weights are 0 the output will be 0 for any input. No bias units is equivalent to bias units are all 0.
In other cases, you can use 0 weights with other activations to output specific nonzero constants; subsequent linear layers can make this constant into any fixed value. For example, logistic(0)=0.5, so you can consider linear combinations of 0.5.
Yes. This happens even when people don't want it to. It happens when people train catagorization networks with very unbalanced datasets, then network will often catagorize every input as the most common class in the training set.
It also happens in GANs, with so called "mode collapse", where the generative portion of the network learns to produce a single output that always tricks the adversarial part.