I have a dataset with all zeros and ones. I am doing unsupervised clustering with t-SNE, and I have already seen some structures, which is great. The dataset is on the scale of 10k rows x 1k cols, so not very big. For example, part of it looks like this

c1 c2 c3
0  0  0
0  1  0
1  0  1
1  1  1

Now, I am thinking that c1 and c2 must both be present, which may have some interesting property based on domain specific knowledge. I am thinking of doing something like c1 AND c2, which would end up in a 4th column, then the example becomes like

c1 c2 c3 c4
0  0  0  0
0  1  0  0
1  0  1  0
1  1  1  1

My question is that will such feature engineering add more information to the dataset or not? My intuition is NO, but I can't explain to myself why. Because I thought if that's ok, then we can add all sorts of combinations, which doesn't feel right. I guess linear combination of multiple columns probably won't work, but AND operation is not linear.

Also, is it true that common ML algorithm, either supervised or unsupervised will automatically capture the information of linear combination of existing columns? I remember being taught that during data processing stage, columns should better be decorrelated if possible, which adds to my confusion. Could anyone clarify please?

  • $\begingroup$ For binary variables AND is a straight interaction term. This is a pretty standard tool in modelling. $\endgroup$ – Glen_b Oct 28 '16 at 1:23
  • $\begingroup$ @Glen_b, thanks, do you have any insight on the linear combination part of the questions, too? $\endgroup$ – zyxue Oct 28 '16 at 1:28
  • $\begingroup$ Actually, thinking about it I really don't know enough about t-SNE to be confident of not saying something dumb. $\endgroup$ – Glen_b Oct 28 '16 at 2:12
  • $\begingroup$ In case you're interested in learning t-SNE, this is a great site, distill.pub/2016/misread-tsne $\endgroup$ – zyxue Oct 28 '16 at 2:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.