How to do multivariate machine learning? (predicting multiple dependent variables) I am looking to predict groups of items that someone will purchase... i.e., I have multiple, colinear dependent variables. 
Rather than building 7 or so independent models to predict the probability of someone buying each of the 7 items, and then combining the results, what methods should I look into to have one model that accounts for the relationships between the 7 related, dependent variables (things they can purchase).
I am using R as a programming language, so any R specific advice is appreciated.
 A: Based on your description, it appears a multinomial logistic regression is appropriate. Assuming your outcome is a factor with 7 levels (one of the 7 buying options), then you can quickly predict membership using a multinomial logistic regression model (see ?multinom in the nnet package in R). If your outcome cannot be combined into a factor with 7 levels, then a cluster analysis would be needed to group the items together before fitting the multinomial logistic regression.
A: You could build a random forest where each of your classes is a group of items (i.e. "green apples with farmed strawberries, with 2% milk").  Then, based on the characteristics of the shopper or whatever your predictors are, you can provide a predicted probability of purchase for each group of items. I would use R's randomForest package (https://cran.r-project.org/web/packages/randomForest/index.html) to do this.
A: One option is to obtain frequencies of all the combinations of product purchases; select the few most common combinations; then build a regression model to predict each individual's chosen combination.  E.g., with a binary logistic regression you could conceivably predict purchase of a) White Wine, Brie, Strawberries and Grapes vs. b) Red Wine, Cheddar and Gouda.  With more than 2 such combinations, or if you want to include the category of "none of the above," multinomial logistic regression would probably be the method of choice.  
Note that including just the common combos means you will have more workable numbers of each but that you will be excluding the others, at least from this procedure. I could imagine 7 items creating dozens of combos each chosen by at least a few people.  This is possibly too many categories for your sample size.  Moreover, if a combo were chosen by  just a few people, your model would have very little information to work with.
Another option is to use cluster analysis to arrive at a few sets of items that tend to be purchased together.  With 7 items, you'll probably end up with fewer than 4 clusters, which might make your task easier.  If you try cluster analysis and find the results unworkable, there is no reason why you have to use them:  just go back to the frequency-based approach described above.  In this case, if I read you right, you're looking for the most descriptive and interesting array of categories, and in establishing that, you don't need to worry about degrees of freedom or multiple comparisons or any such concerns that might apply if you were trying out multiple methods in performing some inferential test.
A: I am assuming that you want to analyze situation similar to the following;
Yi = f(X), where f() is a non-linear link and X is a vector of covariates and Yi is the i-th dependent variable, which is ordinal in nature (if it is categorical Yi cant have more than two categories), and say in your model i = 1, 2, ...5 and each of the Yi s is correlated... If so, you can certainly employ Multivariate Probit. R, Mplus and SAS can estimate MVP
In contrast, you have Y = f(X), and Y (notice there is only one Y) is categorical and for example, has N categories so that that choices made over the N categories are exclusive and exhaustive; you need to fit Multinomial Logit model. There is something called multinomial probit as well, simialr to multinomial Logit.
Hope this helps.
Thanks
Sanjoy
