Return to Question

This is eye-opening, and the effect on KNN, for example, is easy to predict, but should the limitations of Euclidean distance in high dimension be a reason for concern in the very common application of multiple regression?

UPDATE: After all the comments it is clear that it is not the number of examples (subjects or observations or rows in the model matrix) that counts towards dimensionality: it is about the number of features (regressors, independent variables, columns in the model matrix) because no matter how many observations, in a model like $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 +\epsilon$, for example, you end up with a plane (2 dimensional).

Let me rephrase the question:

Is the fact that in the computation of a low-dimensional model very, very high dimensional vectors (many elements or observations), $y\in \mathbb R^{\text{huge}}$, are involved present any dimensionality issues regarding the application of Euclidean distances to high dimensions as in the link above?

This is eye-opening, and the effect on KNN, for example, is easy to predict, but should the limitations of Euclidean distance in high dimension be a reason for concern in the very common application of multiple regression?

UPDATE: After all the comments it is clear that it is not the number of examples (subjects or observations or rows in the model matrix) that counts towards dimensionality: it is about the number of features (regressors, independent variables, columns in the model matrix) because no matter how many observations, in a model like $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 +\epsilon$, for example, you end up with a plane (2 dimensional).

Let me rephrase the question:

Why high dimensional vectors (many elements or rows of observations), $y\in \mathbb R^{\text{huge}}$, do not pose dimensionality issues, while many features (regressors or columns) do? Even if the desired model is low-dimensional, its calculation involves distances in high dimensional spaces.

This is eye-opening, and the effect on KNN, for example, is easy to predict, but should the limitations of Euclidean distance in high dimension be a reason for concern in the very common application of multiple regression?

This is eye-opening, and the effect on KNN, for example, is easy to predict, but should the limitations of Euclidean distance in high dimension be a reason for concern in the very common application of multiple regression?

UPDATE: After all the comments it is clear that it is not the number of examples (subjects or observations or rows in the model matrix) that counts towards dimensionality: it is about the number of features (regressors, independent variables, columns in the model matrix) because no matter how many observations, in a model like $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 +\epsilon$, for example, you end up with a plane (2 dimensional).

Let me rephrase the question:

Is the fact that in the computation of a low-dimensional model very, very high dimensional vectors (many elements or observations), $y\in \mathbb R^{\text{huge}}$, are involved present any dimensionality issues regarding the application of Euclidean distances to high dimensions as in the link above?