0
$\begingroup$

Suppose I have a dataset $X$ that contains both numerical and categorical features. For concreteness let's assume that one of the categorical features is a sample's color, and that it has been properly preprocessed via one-hot encoding. A property of $X$ is that about half the samples are described by one color, and the other half by two.

For instance the data could look like 

Color A     Color B      Height (m)     Weight (kg)     other features

Red         Blue         0.5            1               ...
Green       NaN          0.2            1.2             ...
Purple      Red          0.3            0.5             ...
Blue        NaN          0.45           0.75            ...

I was wondering whether it would be possible to predict the most likely second color for monochromatic samples given the information contained in $X$, and if so what is the best way to frame this question as a statistical/machine learning model?

This problem seems related to clustering, although I have a few issues with taking this point of view. If I were to naively cluster the samples in $X$ that have two colors, then nothing guarantees that the clusters would be based on the color feature only. In fact clustering would most likely group samples with different colors together, invalidating my goal from the start.

Another point of view would be to treat the monochromatic samples as having missing data. I have heard that Expectation Maximization can be used to replace missing data, but it still does so by clustering data using a mixture model, and I go back to my argument in the previous paragraph.

Any guidance on how to approach this problem, if possible, would be greatly appreciated.

Improve this question
$\endgroup$
11
  • $\begingroup$ I'm confused by your description. If color is a single feature, such that you can dummy-code it, how can a single case have two colors? (Don't use the word "samples" to mean "cases" because in statistics, a "sample" means a set of cases, not a single case.) $\endgroup$
    – Kodiologist
    Commented Jul 1, 2017 at 1:11
  • $\begingroup$ @Kodiologist You can think of it as being two separate features, one of them being primary color and the other secondary color, with the latter being NaN for monochromatic observations. It's then straightforward to dummy-code the combination of the two so that some observations effectively have two colors. I'm intrigued in knowing what the most likely secondary color would be based on the information contained in $X$. $\endgroup$
    – physguy
    Commented Jul 1, 2017 at 1:22
  • $\begingroup$ Is there anything to distinguish a case with primary color A and secondary color B from a case with primary color B and secondary color A? Or is the distinction arbitrary? $\endgroup$
    – Kodiologist
    Commented Jul 1, 2017 at 1:29
  • 1
    $\begingroup$ I am not clear why this is not simply a classification problem to predict the second color. $\endgroup$
    – G5W
    Commented Jul 4, 2017 at 1:12
  • 1
    $\begingroup$ Since you say certain values of color A imply no color B, there does appear to be a relationship? And by any means, I cannot see any relationship to clustering here. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Jul 5, 2017 at 6:12

1 Answer 1

Reset to default
0
$\begingroup$

You could try multinomial logistic regression, see for example Multinomial logistic regression vs one-vs-rest binary logistic regression and search this site. Use the Color variable as a (nominal) outcome variable, and the other variables as predictors.

From the fitted model you will get fitted/predicted probabilities for each color, for each case. Then look at the second largest fitted probability.

As for the problems with missing data, look into https://stats.stackexchange.com/questions/tagged/data-imputation and https://stats.stackexchange.com/questions/tagged/multiple-imputation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.