When performing machine learning, the performance of the machine learning method is dependent on the original function $f$ that we are trying to learn (let's forget for a moment the non-deterministic error and focus on the properties of $f$).

I have a conceptual question. What would make a function easy to learn? For instance, a linear function can be easily learnt by linear methods such as linear regression and SVM. But in the general case, what would make a function easier to learn for decision trees or a neural network?

I believe feature engineering helps us move to a feature space in which the function we want to approximate is easier to represent. But what properties exactly are we trying to achieve for this function?