I want to make supervised learning on a dataset containing for each observation a list of labels (the predictors) and a list of types to predict. The train dataset looks like this:

  label1 label2 label3   type1  type2
1   book  novel   <NA>  person writer
2    fly   tree   eggs  animal   bird
3  state   <NA>   <NA> country   <NA>
4  music   band  piano   album   <NA>

I know how to apply machine learning when there is only one variable Y to predict, but I was wondering how to do when there are multiple variables Yi. In my case I would basically want to predict a list of types from a list of labels (knowing that the number of labels and types may vary as shown in the example).

In a more practical way, I was wondering if I should transform the types into binary variables (there might be more than 100 types) like this:

  label1 label2 label3     person     writer     animal     bird     country     album
1   book  novel   <NA>          1          1          0        0           0         0
2    fly   tree   eggs          0          0          1        1           0         0
3  state   <NA>   <NA>          0          0          0        0           1         0
4  music   band  piano          0          0          0        0           0         1

Is multivariate analysis the field I should investigate? I am a newbie in ML so my question may be naive though... Thanks for any help!


I seem to understand that you intend to change all the factor variables in dummy. I use two packages ( helpRFunctions, mlr) and a FOR loop.

t <- list.df.var.types(my.data)

Then take the t$factor and the library mlr and use it the cicle FOR to turn all the factors in dummy (with the same cycle you can create regression groups, 1 group for each dummy group ...)


for (i in 1:length (t$factor)) {
  data <-  createDummyFeatures(data, cols =t$factor[i], method = "reference"

Method "reference" exclude the first dummy.

  • $\begingroup$ Thanks Luigi for the tips. My problem is not really to tranform my categorical variables into binary (I did it with the cast funtion in R), but to process the learning to make a multi-label prediction (anyway thanks for the helpRFunctions, I didn't know it). And indeed the mlr package seems to be exactly what I want for the learning (r-bloggers.com/multilabel-classification-with-mlr)! $\endgroup$ – Tau Jan 3 '18 at 14:29

Yes, multivariate analysis is usually what the area is called in statistics. More specifically you are probably interested in multivariate (multiple) regression when you want to predict $E(\mathbf{Y}| \mathbf{X})$, for some vector $\mathbf{Y}$ given some data $\mathbf{X}$. Note that this problem of calculating the conditional expectation (i.e. prediction) is sometimes called regression in statistics even when $\mathbf{Y}$ is a categorical variable (and would usually be called classification in machine learning). In machine learning, this area problem is sometimes called multitask learning.

Yes, you can transform categorical variables in this way, it's called one-hot encoding in machine learning, and it's standard preprocessing step in many ML libraries.

Finally, I am not an expert about appropriate R packages but I found one party that implements CART-like tree with support for multivariate responses. You might find other useful answers on the CRAN task view, or discussed in this question here (although the answers are not super helpful). Otherwise I'd suggest maybe asking a more specific R related question here.

  • $\begingroup$ Thanks @Machine epsilon for your answer. It seems that multivariate analysis is often associated with regression. In my case I want to make classification. May I apply any classification algorithm to my multivariate problem (svm, random forest...)? In other words, do I need specific packages (I work in R) for multivariate? $\endgroup$ – Tau Jan 3 '18 at 11:12
  • $\begingroup$ I don't know if these will work out of the box for you in R, but I suspect they may not. I have tried to update the answer to better reflect your comments. I'd suggest also adding the R tag to your question. $\endgroup$ – MachineEpsilon Jan 3 '18 at 12:57
  • $\begingroup$ Thanks again, there are some interesting packages in the links you provides, I will check this to make a choice (compare to mlr package). $\endgroup$ – Tau Jan 3 '18 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.