Is it acceptable to use multilabel classification data to predict a continuous score? If a model is given a multilabel classification problem is it appropriate for it to then, as opposed to just predicting the given labels, use those labels to create a scoring scale of ordinal classification? For example, classifying how effective a drug is to treat a disease. I have training data with 4 labels (definitely effective, likely effective, possibly effective, and not effective). Can I encode these 4 labels to be numbers then get models to perform ordinal regression? 
Or would it be more appropriate to have a model perform multilabel classification and take the probability calculated for each drug for the 'definitely effective' label and deem that as a score?
I am new to machine learning, so apologies if my question is rooted in error. For more information if useful, currently I am trying to compare logistic regression, random forest, gradient boosting and deep neural network.
 A: I'm right alongside you in the learning process, but I'll give my two cents anyways.
I believe both methodogies would technically work, and you'd end up with either a score or a probability distribution for each label... but I believe it's a matter of interpretability and use case.
Intuitively, I'm partial to the latter suggestion of probabilities of each label.
I can imagine two use cases where you'd pick opposite approaches.
If you wanted to compare the predicted effectiveness of various potential new treatments, predicting a continuous "score" would allow you to more easily visualize treatments together in a scatterplot, for example.
However if you are building a model to interpret the individual effectiveness of new treatments, I think the class probability distribution would more clearer communicate the risk/benefit of a single treatment.
One downside to the continuous score metric is that a score of "4.0" might mistakenly imply that a treatment is "100% effective all the time", when in reality the definition of "definitely effective" might very greatly according to the expected patient outcome without treatment.
A: Ordinal regression will not treat the four categories as numbers; it doesn't use the numerical values but only the order of the categories. They can however be interpreted as implicitly creating and predicting a continuous score, though this is different from the numbers you choose to encode the levels in advance.
The important thing here is that ordinal regression uses the ordinal information in the data, which looks appropriate for your application. Multilevel classification will not use the information of order in your scale, and will therefore lose some information. This is an advantage of ordinal regression.
On the other hand, ordinal regression comes with model assumptions about how the outcome is related to the x-variables. Particularly it implies that the outcomes are on average monotonically related with a certain linear combination of the x; in fact even the functional form of the relation is assumed. Despite your outcome being ordinal, there is no guarantee that these assumptions are fulfilled. The real situation may be close enough to them that the use of ordinality information that you have in ordinal regression may be a good thing, but it may also be that the relationship is non-monotonic or otherwise "irregular" in a way that a flexible multilabel classification method fits better (these also come with implicit assumptions but are still often more flexible).
One issue here is that you may want to compare approaches using cross-validation and the like. In this case it is important to use a loss function that takes the ordinal nature of your data into account by assigning lower loss for example if you classify a true "3" as "2" rather than "1". Standard loss used in classification will only differentiate between correct and wrong category, and multilabel classification may look better than ordinal regression according to such loss functions even in cases in which there is actually an advantage using ordinal regression because it at least has a better chance of getting "close" if it gets things wrong.
