1
$\begingroup$

If by some prior, it is known that some features are more important than the others, how can this be reflected using feature engineering?

For example, to encode an article into a vector, it is known that the title of article contains more important information than its body, how would one reflect it when using tf-idf or some other word embedding techniques?

$\endgroup$
1
$\begingroup$

I don't have a full answer and will be happy to learn from new answers.

The strictest way to emphasis some features is to provide only them. Other ideas I have are algorithm dependent.

There are algorithms in which weight can be given to a feature (e.g., linear regression, logistic regression). Of course, the best will be to compute the proper weight analytically. In some cases we choose not to do so due to computational cost and use search algorithms like gradient decent. Initialising theses feature with higher weight can make the search more efficient.

In random forest we can give higher probability to these features to be chosen. In graphical models we can start with a model based on them.

In neural networks we can connect them more than other feature. Besides the weight potential, it will enable them to interact in more places.

In some implementations of trees, we can set them to the root of the tree and continue building it from there.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.