Apologies if this belongs to
Data Science instead of here (I can move the question) but this seems related to the math aspect more than ML. In our course we just saw how adding polynomial features to a dataset may improve the model performance. I understand that when there is one single variable, as this allows to perform a linear regression by somehow considering the each power of the variable as an unknown making the equation linear instead of say quadratic for the regression. However, we saw that this can be done for more than one, but not necessarily all, variables/features in a dataset.
- Is the point of adding such features to better fit a model to exponential target variables?
- Which feature(s) to raise to a power and which not to raise?
- What is the point of raising one feature (ex. air temperature) but not another (ex. wind speed)?
- To which power(s) - 2, 3, etc. - should we do this? Is there a way to determine this except by experimenting?
- Can we add a feature risen to degree 3 without the degree 2 for ex. $x$ and $x^3$?
Related to question 2:
Which features should I choose to create polynomial features?