# Feature Engineering: Should I drop features that can be calculated using other features?

In feature engineering, should I drop all features that can be calculated using other features?

For example, let us say that we have this dataset:

f1    f2    f3        f4     target
1     4     0.25      5      9
3     6     0.5       9      8
5     8     0.625     13     12
2     8     0.25      10     15


In this dataset,

f3 = f1 / f2

f4 = f1 + f2

• Should we drop f3 and f4?

• Does the decision depend on the size of the dataset?

• Does the decision depend on the algorithm used (Random Forest, Neural Network, etc)?

• Does the decision depend on the machine-learning type: regression, classification, etc?

• Does the decision depend on the formula that links the features? For example, the formula that links f3 to f1 and f2 is simple division.

If you use algorithms that are based on linear combinations of features ($$\beta_1 f_1 + \beta_2 f_2 + \dots$$), like generalized linear models (including linear regression, logistic regression), or densely connected neural networks, then it doesn't make sense to include variables like $$f_4 = f_1 + f_2$$, since this is already a build-in feature of the model. Moreover, you would get into problems with multicollinearity (by definition, such feature is linearly dependent on others). Including features such as $$f_5 = f_1 f_2$$ means using interaction terms and is commonly used in statistics and $$f_3 = f_1 / f_2$$ is just an interaction in disguise.