The following is my understanding of what happens: if I take a "two dimensional problem" e.g. I have $X$ as inputs and Y as the outcome and I add a feature $x^2$. This gives a problem an additional dimension and the linear fit on the $x$ and $y$ values define a line as well as the linear fit on $x^2$ and $y$ values and the two lines define a plane which is the best fit. Is this correct? How does this translate back to the 2 dimensional space? Does this somehow show up in two dimensions as curvy? How?
This is a piece of a plane in 3D.
Here is the same plane with coordinates shown and a set of points selected along its $x$ axis.
The third coordinate is used to plot the squares of these $x$ values, producing points along a parabola at the base of the coordinate box.
A vertical "curtain" through the parabola intersects the plane at all the points directly above the parabola. This intersection is a curve.
A polynomial model supposes the response $y$ (graphed in the vertical direction) differs from the height of this plane by random amounts. The values of $y$ corresponding to these $x$ coordinates are shown as red dots.
Consequently, the $(x,y)$ points lie along a curve--this projection--rather than a line, even though the model of the response is based on the plane originally shown.
When the explanatory variables clearly lie on a curve, the responses will appear to lie on a curve, too.
If you have a single independent variable x and a single dependent variable y, then "y = f(x)" is usually considered as two dimensional even if the relationship between these two variables is complicated. As a hypothetical example, if an experimental model is "pressure = a * temperature + b * log(temperature) - c * sine(temperature)" there are only two variables, temperature and pressure. For this reason, such a relationship could be plotted as a curved line on a flat plane.
If the model had two independent variables, such as "pressure = a * log(temperature) - b * exp(altitude)", this has the form of "z = f(x,y)" and could be plotted as a 3D surface.