1
$\begingroup$

I am able to implement supervised learning algorithms when the output label ($Y$) is one dimensional. Just in case, the output label is multidimensional ($Y$= [$y_1$ $y_2$ ... $y_s$]), I believe that for such cases, we have to have $s$ different models with $i^{th}$ model predicting $y_i$ from the input data.

Am I right with this interpretation ? Any better alternative ?

Let's assume I am implementing linear regression algorithm on a labelled data with the output label being $s$ dimensional.

$\endgroup$
2
$\begingroup$

For linear regression, a muti-label output is the same as multiple linear regression models - each has its own coefficients. But for other models, such as a neural network, one model with several outputs is not the same as several independent models. In such a case, if you believe the labels depend on some common underlying structure then it is better to use one model with multiple outputs.

$\endgroup$
  • $\begingroup$ could you please explain the last line of yours "In such a case ... with multiple outputs" Why not this sentence hold good for linear regression as well. I mean, in linear regression also, y1 and y2 are dependent on 'some common underlying structure' that we call as features. I am getting confused with 'some common underlying structure'. Could you please elaborate on this $\endgroup$ – Upendra Pratap Singh Jan 28 '18 at 9:09
  • $\begingroup$ @UpendraPratapSingh in neural networks, the output layer depends on the last hidden layer, and that layer depends on the layer before it, etc. It is possible for the whole network to learn to represent the data and only the output layer distinguishes between the different outputs. This is not the case in linear regression. $\endgroup$ – elliotp Jan 28 '18 at 9:24
  • $\begingroup$ So 'some common underlying structure' refers to the abstract representation of the data at the last hidden layer Right? $\endgroup$ – Upendra Pratap Singh Jan 28 '18 at 9:29
  • $\begingroup$ @UpendraPratapSingh it can be several of the last layers. For example, a convolutional neural network might learn to represent natural images well in a low dimensional space, and then the final two layers learn to classify animals. Cat or not cat would be one output, dog or not dog would be a second output, etc. Most of the work being done is common to all the animals. $\endgroup$ – elliotp Jan 28 '18 at 9:35
  • $\begingroup$ Right... but with this explanation of yours, even the features in linear regression are representing the data and the output labels (y1 as well as y2) are dependent on these features (through coefficients). So, now, what is it that allows us to choose separate models for different y's and what makes it absolutely necessary for us to go with a single model with multiple outputs ? I believe that this distinction is an important issue for the entire community of machine learning practitioners. Kindly educate me on this. By the way. thanks for showing a genuine interest in my query. $\endgroup$ – Upendra Pratap Singh Jan 28 '18 at 9:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.