Feature Interaction and its applications?

Question

What is a Feature Interaction?

Are Feature Interactions used for Feature Selection or Feature Generation?

Answer 1

Quoted from the OP's link:

5.4 Feature Interaction When features interact with each other in a prediction model, the prediction cannot be expressed as the sum of the feature effects, because the effect of one feature depends on the value of the other feature. Aristotle's predicate "The whole is greater than the sum of its parts" applies in the presence of interactions.

5.4.1 Feature Interaction? If a machine learning model makes a prediction based on two features, we can decompose the prediction into four terms: a constant term, a term for the first feature, a term for the second feature and a term for the interaction between the two features. The interaction between two features is the change in the prediction that occurs by varying the features after considering the individual feature effects.

I would like to be a little pedantic and alter the last sentence above:

The interaction between two features is the change in the prediction that occurs by varying the features while considering the individual feature effects.

Another way to think about an interaction is that it occurs when the effect of one feature depends on the value of another feature. Note that interaction are a natural result of considering the general model:

$$ Y = f(x,Z)$$

where $x$ is a matrix of continuous explanatory variables (features) and $Z$ is a random variable which we can think of as normally distributed around zero, but it does not have to be. If we expand this around $x_0$, with a 2nd order taylor series we obtain:

$$Y \approx \beta_0 + \sum_{i = 1}^{p} \beta_{i}(x_i-x_{i0}) + \sum_{i = 1}^{p}\sum_{j = 1}^{p} \beta_{ij}(x_i-x_{i0})(x_j-x_{j0}) + \left( \sigma + \sum_{i = 1}^{p} \gamma_i(x_i-x_{i0}) \right) Z + \sigma Z^2 $$

where the 3rd term contains cross products of two linear terms - which are the interactions.

Are Feature Interactions used for Feature Selection or Feature Generation?

I would consider interactions as Feature Generation, and they are a very useful way of modelling a natural form of nonlinearity.

Edit: Of course, once you have generated the interaction feature, there is then the question of whether to include it in your model, and that is where feature selection comes into play.

Answer 2

Feature interaction seems to be just new (machine learning?) terminology for plain old interaction. As such there is already many posts at this site, see this list.