Utility of feature-engineering : Why create new features based on existing features? I often see people create new features based on existing features on a machine learning problem.
For example, here : https://triangleinequality.wordpress.com/2013/09/08/basic-feature-engineering-with-the-titanic-data/ people have considered the size of a person’s family as a new feature, based on the number of brothers, sisters, and parents, which were existing features.
But what the point of this ? I don't understand why the creation of correlated new features is useful. Isn't it the job of the algorithm to do that on its own ?
 A: It is true that some of the machine learning models have the ability to handle the non-linearity and interaction between variables, however, depends on the situation, I see three reasons it becomes necessary.


*

*Some models like linear regression don't handle non-linearity automatically, in that case, you need to create extra features to help. For example below: if you have the following dataset that all the $Y = 1$ of the target variable clustered at the center of a circle like area. 



If you are given only two features, $x_1$ and $x_2$. A simple linear model of $y = x_0 + c_1x_1 + c_2x_2$ will not find any way to classify the target variable. So, instead, you need new quartic features to capture the non-linearity: $y = x_0 + c_1x_1^2 + c_2x_2^2$.


*If you know in advance that some features (from business knowledge or experience), it may help create them to speed up the runtime of the model and make it easy for your model. For example, in your example of the Titanic data and if you are using a decision tree classification model. If you know that old ladies (age & gender) are more likely to survive, by creating a single feature that captures the information, your tree can make one split on the new variable instead of making two split on the two variables. It may speed up the computation time if you know in advance that the feature is significant.

*In the real world, you won't get a single dataset like Kaggle provides. Instead, you get information from all over the place. For example, if you want to predict customer attrition for an online retail company like Amazon, you have customer demography info, purchase transaction info. You need to generate a lot of feature from different sources, in this case, You will find a lot of useful features can be obtained/aggregated from the transaction level. As Andrew Ng puts it: Often times, the ability to do feature-engineering defines the success or failure of a machine learning project.
A: The most simple example used to illustrate this is the XOR problem (see image below). Imagine that you are given data containing of $x$ and $y$ coordinated and the binary class to predict. You could expect your machine learning algorithm to find out the correct decision boundary by itself, but if you generated additional feature $z=xy$, then the problem becomes trivial as $z>0$ gives you nearly perfect decision criterion for classification and you used just simple arithmetic!

So while in many cases you could expect from the algorithm to find the solution, alternatively, by feature engineering you could simplify the problem. Simple problems are easier and faster to solve, and need less complicated algorithms. Simple algorithms are often more robust, the results are often more interpretable, they are more scalable (less computational resources, time to train, etc.), and portable. You can find more examples and explanations in the wonderful talk by Vincent D. Warmerdam, given on from PyData conference in London.
Moreover, don't believe everything the machine learning marketers tell you. In most cases, the algorithms won't "learn by themselves". You usually have limited time, resources, computational power, and the data has usually a limited size and is noisy, neither of these helps.
Taking this to the extreme, you could provide your data as photos of handwritten notes of the experiment result and pass them to the complicated neural network. It would first learn to recognize the data on pictures, then learn to understand it, and make predictions. To do so, you would need a powerful computer and lots of time for training and tuning the model and need huge amounts of data because of using a complicated neural network. Providing the data in a computer-readable format (as tables of numbers), simplifies the problem tremendously, since you don't need all the character recognition. You can think of feature engineering as a next step, where you transform the data in such a way to create meaningful features so that your algorithm has even less to figure out on its own. To give an analogy, it is like you wanted to read a book in a foreign language, so that you needed to learn the language first, versus reading it translated in the language that you understand.
In the Titanic data example, your algorithm would need to figure out that summing family members makes sense, to get the "family size" feature (yes, I'm personalizing it in here). This is an obvious feature for a human, but it is not obvious if you see the data as just some columns of the numbers. If you don't know what columns are meaningful when considered together with other columns, the algorithm could figure it out by trying each possible combination of such columns. Sure, we have clever ways of doing this, but still, it is much easier if the information is given to the algorithm right away.
A: Well, if you plan to use a simple, linear classifier, it makes perfect sense to generate new features which are a non-linear function of the existing ones, specially if your domain knowledge indicates you the resulting feature will be meaningful and informative. Note that a linear classifier cannot consider those complex features unless you explicitly provide them.
Ideally, If you use a sufficiently powerful nonlinear classification algorithm it should be able to create a decision boundary which considers arbitrary non-linear transformations of the input features if they are informative for classification. However, in practice most non-linear classifiers just look at some type of transformations. For instance, a polynomial kernel SVM will consider polynomial interactions between features, but maybe a more informative feature can be created by applying other types of transformations... 
In short, if domain knowledge indicates that a hand-crafted non-linear combination of features might be informative, it makes sense to add that into the existing set of features.
