1
$\begingroup$

I recently realized, that feature engineering (designing input vectors for machine-learning algorithm) is one of the most complicated tasks when applying known algorithms (for example kernel perceptron). My question is: is it okay for scalars in the vector to be of different nature? For example, if your vector is (width, height, length) that seems okay to me, because all these three parameters are of the same nature and it's really logical to be there together. I know, all these terms are very intuitive (as is my concern) but I hope I elaborated enough.

In my example, there are mixed features, for example: whether or not A action was performed, was action B performed after action C or not, to what extent was action D performed etc...

$\endgroup$
2
$\begingroup$

Tipically, having all features from the same nature is very rare. You mostly have various and very different categories of data. Take for instance patient data. It usually consists of age, weight, height, and various lab results or other diagnostics - all very different from each other.

The machine learning algorithms aren't picky about the nature of each feature (attribute). They just look at the raw values and try to find some patterns or infer some knowledge/rules from them. When using generic machine learning algorithms, they usually don't even understand the meaning of the attribute.

You can look at some sample datasets at the UCI repository to get an idea of how this is done in practice. It would also help if you would say more about your problem and the possible features you could use.

$\endgroup$
4
  • $\begingroup$ Very true, with the caveat that some algorithms benefit from feature scaling. $\endgroup$ – image_doctor Apr 24 '15 at 9:44
  • $\begingroup$ True. But then again, leaving out a potentially beneficial feature just because it is not of the same nature, is not good either. As in everything else, it has it's pros/cons. $\endgroup$ – alesc Apr 24 '15 at 9:48
  • $\begingroup$ I added some info at the end of question $\endgroup$ – nicks Apr 24 '15 at 10:11
  • $\begingroup$ One thing to be careful of is if you're using regularization (e.g. lasso) you should make sure all your features are on the same scale as otherwise you'll penalize those with higher magnitude more $\endgroup$ – kezz_smc Apr 24 '15 at 12:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.