Intro: suppose we have $n$ observations with $m$ features, represented by a $n\times m$-matrix $X$, and two specific points $x,y\in\mathbb{R}^m$, and we are interested in distances between $X$ and $x,y$, respectively. Since the features may be of different orders of magnitude, we standardize every feature of $X$, then calculate respective z-scores for coordinates of $x$, $y$ in order to be scale-independent (or to measure separation with respect to standard deviations of each feature). Then we may apply some particular distance function (e.g. euclidean, Mahalanobis) to compute and compare $d_X(X,x)$ and $d_X(X,y)$.

(Here $d_X$ means that the distance is calculated after the standarization with respect to $X$.)

Say we face the opposite task, i.e. given two series of observations ($n\times m$-matrices) $X$, $Y$ and a specific point $x\in\mathbb{R}^m$, decide which one of $X,Y$ is closer to $x$, in the scale-independent sense sketched above. Two strategies seem to be valid:

  1. standardize $X$, calculate z-score of $x$, then calculate distance $d_X(X,x)$; standardize separately $Y$, again calculate z-score of $x$, then calculate distance $d_Y(Y,x)$; in the end compare $d_X(X,x)$ and $d_Y(Y,x)$

  2. first concatenate $X$ and $Y$ to a full $2n\times m$-matrix $Z$ of all observations, standardize $Z$, calculate z-score of $x$, compare $d_Z(X,x)$ and $d_Z(Y,x)$.

Question: which strategy is appropriate? My guess is that, assuming the reasoning from Intro, the first one seems to be more natural.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy