Questions about adding polynomial features to a dataset for linear regression

Question

Apologies if this belongs to Data Science instead of here (I can move the question) but this seems related to the math aspect more than ML. In our course we just saw how adding polynomial features to a dataset may improve the model performance. I understand that when there is one single variable, as this allows to perform a linear regression by somehow considering the each power of the variable as an unknown making the equation linear instead of say quadratic for the regression. However, we saw that this can be done for more than one, but not necessarily all, variables/features in a dataset.

Questions

1. Is the point of adding such features to better fit a model to exponential target variables?
2. Which feature(s) to raise to a power and which not to raise?
3. What is the point of raising one feature (ex. air temperature) but not another (ex. wind speed)?
4. To which power(s) - 2, 3, etc. - should we do this? Is there a way to determine this except by experimenting?
5. Can we add a feature risen to degree 3 without the degree 2 for ex. $x$ and $x^3$?

Related to question 2:
Which features should I choose to create polynomial features?

Answer 1

Is the point of adding such features to better fit a model to exponential target variables?

Sometimes. Sometimes, there is good reason to expect a quadratic effect to be present (free projectiles is a good example). Other times, it may be that a higher order polynomial better predicts some outcome than a linear one. If you validate the model correctly, that seems fine too.

Can we add a feature risen to degree 3 without the degree 2 for ex.

Technically, $x^2$ is an interaction between $x$ and itself, and there is some guidance that if you have an interaction between two variables then all lower order terms should appear in the model too (but that may just be for inference, prediction is a whole other beast).