It is clear to me that adding polynomial features fixes high bias. So far so good. But there is also the claim that adding more (non polynomial) features fixes high bias. I don't see why. In my humble opinion it will not fix it since linear regression will stay linear regression if a new, non-polynomial feature is added. With one feature we will have a straight line in a plain, and by adding one feature we will have a straight plane in a 3 dimensional space.

Setup

In a regression setting, consider the issue of model bias (rather than sampling bias or other statistical biases). If the true data generating process looks like $Y = f(X) + \mathrm{noise}$ and our goal is to estimate $f$ well, we might fit a particular statistical model or machine learning algorithm to the available dataset and get an estimate $\hat f$. For some procedures, perhaps at many points $X=x_0$ it turns out that $\mathbb{E}(\hat f(x_0)) \approx f(x_0)$, and we say the model bias is low. In other words: on average across many repetitions of taking a sample and fitting the model, the fitted models tend to match the true regression function across most of the range of $X$.

For other procedures, perhaps it turns out that $\mathbb{E}(\hat f(x_0)) \neq f(x_0)$ at many points $X=x_0$, and we say the model bias is high(er). This concept is often illustrated with the example of a nonlinear $f$ and a single feature $X$. For instance, if the true function is $f(X) = X^2$ on the range $X \in (-1, 1)$, but we try to fit $\hat f$ with a simple linear regression, there's no way a straight line can match the true $f$ for most of the relevant range of $X$, not even in expectation. So a simple linear model would have relatively high model bias for the true regression function here, compared to (for instance) polynomial regression or spline models which could have low or no model bias.

Question

It is clear to me how adding polynomials (or other nonlinear transformations) of the features could reduce model bias. So far so good. But there is also the claim that adding more features (entirely new measurements, not transformations of features already in the model) can also reduce model bias. I don't see why. In my humble opinion it will not fix it since linear regression will stay linear if a new feature is added. With one feature we will have a straight line in a plane. By adding a totally new feature, we will still have a "straight" fitted function (a plane with no nonlinearities) in a 3 dimensional space.

So in what sense is it possible for model bias to be reduced when we add new features, not just nonlinear transformations on existing features?