I'm using two 3-layer feedforward multi-layer perceptrons (MLPs). With the same input data (14 input neurons), I do one classification (true/false), and one regression (if true, "how much")¹. Until now, I've lazily used Matlabs patternnet and fitnet, respectively. Lazily, because I haven't taken the time to really understand what's going on — and I should. Moreover, I need to make the transition to an OSS library (probably FANN), that will likely require more manual setup than the Matlab NN Toolbox. Therefore, I'm trying to understand more precisely what's going on.
The networks created by
fitnet are nearly identical: 14 inputs neurons, 11 hidden neurons, 1 target neuron (2 for the
fitnet, but only 1 piece of informatio). But, they're not completely identical. The differences by default are:
- Matlab uses a Scaled conjugate gradient backpropagation for the classification network (
patternnet), and Levenberg-Marquardt backpropagation for the regression network (
- The classification network uses a hyperbolic tangent sigmoid transfer function between the input and the hidden layer, and between the hidden and the output layer. The regression network (
fitnet) uses the Hyperbolic tangent sigmoid transfer function between the input and the hidden layer, and a purely linear transfer function between the hidden and the output layer.
Should those differences be?
What kind of backpropagation functions are optimal for classification, and what kind for regression, and why?
What kind of transfer functions are optimal for classification, and what kind for regression, and why?
¹The classification is for "cloudy" or "cloud-free" (2 complementary targets), the regression is for quantifying "how much cloud" (1 target).