Intro: suppose we have $n$ observations with $m$ features, represented by a $n\times m$-matrix $X$, and two specific points $x,y\in\mathbb{R}^m$, and we are interested in distances between $X$ and $x,y$, respectively. Since the features may be of different orders of magnitude, we standardize every feature of $X$, then calculate respective z-scores for coordinates of $x$, $y$ in order to be scale-independent (or to measure separation with respect to standard deviations of each feature). Then we may apply some particular distance function (e.g. euclidean, Mahalanobis) to compute and compare $d_X(X,x)$ and $d_X(X,y)$.
(Here $d_X$ means that the distance is calculated after the standarization with respect to $X$.)
Say we face the opposite task, i.e. given two series of observations ($n\times m$-matrices) $X$, $Y$ and a specific point $x\in\mathbb{R}^m$, decide which one of $X,Y$ is closer to $x$, in the scale-independent sense sketched above. Two strategies seem to be valid:
standardize $X$, calculate z-score of $x$, then calculate distance $d_X(X,x)$; standardize separately $Y$, again calculate z-score of $x$, then calculate distance $d_Y(Y,x)$; in the end compare $d_X(X,x)$ and $d_Y(Y,x)$
first concatenate $X$ and $Y$ to a full $2n\times m$-matrix $Z$ of all observations, standardize $Z$, calculate z-score of $x$, compare $d_Z(X,x)$ and $d_Z(Y,x)$.
Question: which strategy is appropriate? My guess is that, assuming the reasoning from Intro, the first one seems to be more natural.
