12
$\begingroup$

Feature engineering is often an important component to machine learning (it was used heavily to win the KDD Cup in 2010). However, I find that most feature engineering techniques either

  • destroy any intuitive meaning of the underlying features or
  • are very specific to a particular domain or even particular types of features.

A classic example of the former would be principal component analysis. It would seem to me that any knowledge that a subject-matter expert would have about the features would be destroyed by converting those features to principal components.

Contrast that with a simple technique of converting a date into features for "day of month" and "day of week." The underlying meaning is still retained in the new features, but obviously this particular technique only applies for dates and not arbitrary features.

Is there any standard body of feature engineering techniques that do not destroy the meaning of the underlying features while also being applicable to arbitrary domains (or at least a wide variety of domains)?

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ PCA can sometimes be used to find intuitive meanings for the features - e.g. eigenfaces. $\endgroup$ – tdc Feb 14 '12 at 13:10
  • $\begingroup$ Can you give (more) examples of the data you have in mind ? if you can be more specific about your application (even some arbitrary example) it will be easier to give (more) accurate answer. $\endgroup$ – Dov Feb 17 '12 at 18:18
  • 1
    $\begingroup$ @Dov Well the whole point is that (ideally) I'd like something that could work for just about any structured, tabular dataset (one that has datapoints and features). So this could be sales data, financial data, drug discovery data, baseball data, etc. $\endgroup$ – Michael McGowan Feb 17 '12 at 19:28
7
+50
$\begingroup$

I am aware of one decomposition method (but maybe there are more...) that can be useful in a scenarios like you describe. It is like 2D-PCA - a high order decomposition method where the decomposition (i.e the factors) have some meaning. You can see examples and read about it here and here and try to here

|cite|improve this answer
$\endgroup$
  • $\begingroup$ + please forgive me that I am not a native English speaker :) $\endgroup$ – Dov Feb 8 '12 at 6:34
  • $\begingroup$ From what I was told, last step of PCA should be attempt to find meaning for principial componenet. $\endgroup$ – jb. Feb 8 '12 at 19:48
5
$\begingroup$

Recent deep learning methods using Restricted Boltzmann Machine have shown nice features on several data types (audio, images, text).

Since these methods create a generative model, you can often generate really nice samples from the model.

Check out Hinton's publications. http://www.cs.toronto.edu/~hinton/

These methods aren't totally general (run same code on every data), but the underlying model is usually similar.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.