# Multiple (not independent) response variables in machine learning

Question: How to predict the percentage of people with age < 18, 18-65 and > 65 who visit a webpage using machine learning in R? Since these percentages sum to 100 for all observations, they are not independent.

Data: Contains 400 observations of 16 independent variables (TF-IDF values of 16 keywords for all pages) along with % of people with age < 18, 18-65 and > 65. This can be split into test and train set.

My idea: Recode the output variable as a factor (1: percentage of < 18 age group is highest, 2: percentage of 18-65 age group is highest, 3: otherwise). Build a generative classifier (like Naive Bayes) which predicts the probability of belonging to a each of the classes, which can be converted into percentage.

The problem: Variance of the output variable decreases recoding. This may give inaccurate results.

One potential issue with this approach is that the percentages might exceed 100 in some instances. You could try to come up with some aggregation rule, or try to train a model to predict directly a 3 dimensional outcome (essentially you're trying to predict where to fall in the portion of a $100*100*100$ cube below the $x+y+z=100$ plane). It might be possible. I have found this reference about regression with a two-dimensional output variable which seems useful. The excellent lectures about spatial stats by Douglas Nychka from the NCAR might also be helpful. Some R code is also available.
• If there are data on actual ages, using age as a continuous outcome variable would be best. If only the 3 age classes are available as an outcome variable, multinomial regression would be a useful way to proceed; it generalizes the True/False dichotomous outcome variable in logistic regression to multiple exclusive categories. Follow the multinomial tag on this site; there are readily available tools for performing such analyses. – EdM Aug 10 '15 at 16:24