Assume i want to apply nonlinear regression to two output variables with multilayer perceptrons. Is there difference between using a MLP for each regression with single output and using a single MLP with two outputs? I guess weight updates of the second layer is only affected by corresponding output unit that weights associated but first layer is updated according to backpropagated error of hiddens units from all output units.Still, nonlinearity is not introduced for first layer in mlps so i guess these two approaches will be same?

Secondly, the relation between output variables(i.e if they are correlated such as human weight and height) would make a difference upon the answer of the first question?


Two separate MLPs with the same inputs and the same number of hidden layers can be viewed as a single joint MLP where some connections have been pruned, specifically where all the weight matrices (other than the input-to-first-hidden) have a block-diagonal structure.

Whether training two separate MLPs or a joint-output one will be better is an empirical question which depends on your task that should be answered by model selection over a validation set.

In order to train the joint-output MLP you'll need a loss function that reasonably trades off the error between the two outputs (in principle, the MLP may become very accurate on one output by sacrificing accuracy on the other one).
However, the joint-output MLP may be able to exploit correlations between the outputs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.