Yes, normalisation/scaling is typically recommended and sometimes very important. Especially for neural networks, normalisation can be very crucial because when you input unnormalised inputs to activation functions, you can get stuck in a very flat region in the domain and may not learn at all. Or worse, you can end up with numerical issues.
One very obvious reason is that you need to tune (but you don't) the weight initialisations in the network according to the input range corresponding to that weight, e.g. let $x_1,x_2$ be two distinct features and $w_1,w_2$ be the corresponding weights. Also let the range of the feature be as follows: $x_1\in[0,1000],x_2\in[0,1]$. When you initialise $w_i$ with numbers within $[-1,1]$ for example, it won't mean the same for $x_1$ and $x_2$. Probably, the sum $w_1x_1+w_2x_2$ will be dominated by $w_1x_1$ and you won't see the effect of $w_2x_2$ for some time unless you're very lucky, and learning will be hindered significantly until the network is finally able to learn what $w_1$ should have been in the first place.
Doesn't normalization require that data conforms to the normal parametric distribution?
No, normalisation has nothing to do with normal distribution. One form of normalisation, called standardising, which is subtracting the mean and dividing by the deviation is very common in the literature and typically used for converting a normal RV into standard normal RV. Although the idea may stem from normal distributions, the operation has nothing to do with normal distribution.