My understanding is that the curse of dimensionality implies that we need an exponential amount of data with respect to the number of features we include in our model. Is this correct?
If so, what does "we need" mean? Does it imply that we need at least that many data points to ensure we don't make a mistake?...to negate the effects of the dimensionality?..to ensure we've hit a global optimum?...something else?
Most important question(s) to me:
What specifically are the implications of the the curse of dimesionality for ordinary least squares linear regression?
If we are performing an OLS linear regression with p covariates, do we need 2^p data points?
I've read about rules of thumb for determining how many data points you need for OLS regression with respect to the number of covariates included in the model, and I know that the answer entirely depends on the properties of the data, but I'm trying to get a better understanding for how the curse of dimensionality plays a role in/affects this.