7
$\begingroup$

What is a Feature Interaction?

Are Feature Interactions used for Feature Selection or Feature Generation?

$\endgroup$
1
  • 1
    $\begingroup$ Are you referring to interaction terms such as in linear regression? $\endgroup$ – user2974951 Mar 1 at 7:26
3
$\begingroup$

Quoted from the OP's link:

5.4 Feature Interaction When features interact with each other in a prediction model, the prediction cannot be expressed as the sum of the feature effects, because the effect of one feature depends on the value of the other feature. Aristotle's predicate "The whole is greater than the sum of its parts" applies in the presence of interactions.

5.4.1 Feature Interaction? If a machine learning model makes a prediction based on two features, we can decompose the prediction into four terms: a constant term, a term for the first feature, a term for the second feature and a term for the interaction between the two features. The interaction between two features is the change in the prediction that occurs by varying the features after considering the individual feature effects.

I would like to be a little pedantic and alter the last sentence above:

The interaction between two features is the change in the prediction that occurs by varying the features while considering the individual feature effects.

Another way to think about an interaction is that it occurs when the effect of one feature depends on the value of another feature. Note that interaction are a natural result of considering the general model:

$$ Y = f(x,Z)$$

where $x$ is a matrix of continuous explanatory variables (features) and $Z$ is a random variable which we can think of as normally distributed around zero, but it does not have to be. If we expand this around $x_0$, with a 2nd order taylor series we obtain:

$$Y \approx \beta_0 + \sum_{i = 1}^{p} \beta_{i}(x_i-x_{i0}) + \sum_{i = 1}^{p}\sum_{j = 1}^{p} \beta_{ij}(x_i-x_{i0})(x_j-x_{j0}) + \left( \sigma + \sum_{i = 1}^{p} \gamma_i(x_i-x_{i0}) \right) Z + \sigma Z^2 $$

where the 3rd term contains cross products of two linear terms - which are the interactions.

Are Feature Interactions used for Feature Selection or Feature Generation?

I would consider interactions as Feature Generation, and they are a very useful way of modelling a natural form of nonlinearity.

Edit: Of course, once you have generated the interaction feature, there is then the question of whether to include it in your model, and that is where feature selection comes into play.

$\endgroup$
7
  • $\begingroup$ Pycaret states that "Creating a new feature through the interaction of existing features is known as feature interaction". Are feature interactions not used for feature selection?? $\endgroup$ – Pluviophile Mar 1 at 9:20
  • $\begingroup$ I thought H-statistic is used to measure feature interactions. And this can be used to determine feature selection. Am I wrong?? $\endgroup$ – Pluviophile Mar 1 at 9:23
  • $\begingroup$ The way I see it, you can only "select" a feature if it exists. An interaction does not exist until you generate it (by multiplying two features together). I'm not sure what H statistic you are referring to ? $\endgroup$ – Robert Long Mar 1 at 9:30
  • $\begingroup$ Friedman's H-statistic $\endgroup$ – Pluviophile Mar 1 at 10:15
  • 1
    $\begingroup$ OK thanks. That doesn't affect my answer $\endgroup$ – Robert Long Mar 1 at 10:33
4
$\begingroup$

Feature interaction seems to be just new (machine learning?) terminology for plain old interaction. As such there is already many posts at this site, see this list.

$\endgroup$
3
  • $\begingroup$ Pardon me. In short, what does the "old interaction" mean?? $\endgroup$ – Pluviophile Mar 1 at 12:12
  • $\begingroup$ Just interaction as used in regression modeling. See any regression text! $\endgroup$ – kjetil b halvorsen Mar 1 at 16:23
  • $\begingroup$ Got it. Thank you $\endgroup$ – Pluviophile Mar 1 at 16:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.