Can ML or DL model automatically pick up "difference" feature?

Question

ML requires manual feature extraction whereas DL doesn't necessarily require feature engineering, since recent advanced models like transformers learn necessary features automatically during training - even the sequence of data.

However, I'm still not sure if it means that the model can automatically pick up the features that help understand the data pattern without any preprocessing whatsoever.

Difference is something very simple yet important when you do any sort of feature engineering. For example you have to take difference of price to get the price change, and you have to take difference of timestamps to get the duration of some logs.

This is very simple:

However, can ML models and DL models automatically "use" this feature without explicitly given?

Say I have some columns like timestamp and price.

Is it absolutely pointless to add duration or price_%change?

There are many ML and DL models but I want to know if there are models that are robust to these feature selections so that the model renders feature engineering useless.

Answer 1

It is just a linear function with parameters $\beta_1 = 1$ and $\beta_2 = -1$

$$ \beta_1 x_1 + \beta_2 x_2 = 1 \times x_1 + (-1) \times x_2 = x_1 - x_2 $$

Neural networks use such linear functions commonly. If you have a simple multilayer feed-forward network, just one of the layers would need to learn the parameters above to calculate the new feature to be used by the next layer. Of course, the network wouldn't learn exactly this, as it is unlikely the learned weights would be exactly $1$ and $-1$ and the result would be passed through the non-linear activation function, but it will be able to learn about the kind of relationship you are describing and use it to learn higher-level patterns.

That said, there are cases where engineering the features by hand makes life easier for the algorithm so it does not need to figure them out.