1
$\begingroup$

The term 'feature function' is very frequently used in the context of machine learning, but I'm still not sure what it really is. Could anyone give the precise definition? Can it be understood as a function that maps from the data-points to the feature vector?

What would be the feature function e.g. in the logistic regression?

$\endgroup$
6
  • 1
    $\begingroup$ Never heard the term 'feature function'. Can you post an example? $\endgroup$
    – Firebug
    Apr 7 '21 at 12:06
  • $\begingroup$ For example in this script: github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/… on page 14 $\endgroup$ Apr 7 '21 at 12:13
  • 2
    $\begingroup$ That must be particular to NLP, but I see something like that most often being called an embedding. It's basically a function that takes a type of information (in this case, words) and translate it into something numerical that an ML algorithm can digest. Embedding functions are not trivial though, there's a lot of thought put into more complex ones. People don't like calling a bag-of-words an embedding due to its unrealistic assumptions and some confusion regarding embedding, but it really is a sentence embedding. $\endgroup$
    – Firebug
    Apr 7 '21 at 12:29
  • 1
    $\begingroup$ NLPer here. Adding to Firebug's great comment, the feature function does not have to be a nontrivial embedding. A feature function is any way of quantifying your input, like counting the number of noun phrases, or the sentence length, or even simple word identity. Examples of feature functions used for four NLP tasks are given in sections 4.1, 5.2, 6.2, and 7.3 of this paper. $\endgroup$ Apr 8 '21 at 13:25
  • $\begingroup$ Does this answer your question? In natural language parsing, what is the feature function? $\endgroup$ Apr 8 '21 at 13:26
0
$\begingroup$

Converted my comment into an answer, in the absence of another answer.

That must be particular to NLP, but I see something like that most often being called an embedding. It's basically a function that takes a type of information (in this case, words) and translate it into something numerical that an ML algorithm can digest. Embedding functions are not trivial though, there's a lot of thought put into more complex ones. People don't like calling a bag-of-words an embedding due to its unrealistic assumptions and some confusion regarding embedding, but it really is a sentence embedding.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.