Output layer of artificial neural networks when learning non-linear functions with limited value range I want to approximate a non-linear function with a limited value range by an artificial neural network (feed forward, back propagation).
Most tools and literature availabe suggest linear functions for the output neurons when doing regressions. However, I know a priori that my goal function is of limited range, therefore is it reasonable to use a function for the output neurons with limited value range, too? To be more explicit: My target function's values are in the range between  0 and 1, but the neural net does predict    occasionally values that exceed this range (e.g. -1.3). Can i prevent the net from doing so, and is it reasonable?
 A: If you use a logistic activation function in the output layer it will restrict the output to the range 0-1 as you require.  
However if you have a regression problem with a restricted output range the sum-of-squares error metric may not be ideal and maybe a beta noise model might be more appropriate (c.f. beta regression, which IIRC is implemented in an R package, but I have never used it myself)
A: I am opposed to cutting values of, since this will lead to an undifferentiable transfer function and your gradient based training algorithm might screw up.
The sigmoid function at the output layer is fine: $\sigma(x) = \frac{1}{1 + e^{-x}}$. It will squash any output to lie within $(0, 1)$. So you can get arbitrarily close to the targets.
However, if you use the squared error you will lose the property of a "matching loss function". When using linear outputs for a squared error, the derivatives of the error reduce to $y - t$ where $y$ is the output and $t$ the corresponding target value. So you have to check your gradients.
I have personally had good results with sigmoids as outputs when I have targets in that range and using sum of squares error anyway.
A: If you know an absolute range for the output, but there is no reason to expect it to have the non-linear characteristic of the typical logistic activation function (i.e. a value in the middle is just as likely as a value near 0 or 1), then you can just transform the output by dividing by the absolute maximum.  If the minimum were not 0, you could subtract the absolute minimum before dividing by the value (maximum - minimum).
So basically don't try to train the neural network to the raw value, train it to the percentile value (0 for minimum, 1 for maximum).
A: "Would it work to use the linear function and simply cut all values below 0 to 0, and values above 1 to 1?"
I believe in many cases the cut-off value should be the percentage split of the training data. Eg if your training data has 13% - 0's and 87%  - 1's, then the cut-off would be 0.13; For example anything 0.13 and below on the output is 0 and anything 0.14 and above is 1. Obviously there is more uncertainty the closer to the cut-off the output provides. It may also help adjusting the cut-off limits especially where the cost of a mis-classification is high. This link may help a little http://timmanns.blogspot.com/2009/11/building-neural-networks-on-unbalanced.html
