How to find nearest neighbours to a dataset? I came across a (maybe simple) problem. Consider I have an "observation" vector of 12 values, called $O=[o_1,o_2,...,o_{12}]$. What would be the best way to compare $O$ to a "theoretical" database of $60000$ vectors ($T_j=[t_{j,1},t_{j,2},...,t_{j,12}], where \ j=1 \rightarrow 60000$) in order to find the nearest neighbors to $O$. In other words, I need to find what indexes among the $60000$ are the closest to $O$.
I performed a Chi-squared approach as follows:
$\chi_j=\sum_{i=1}^{12} (o_i-t_{j,i})^2$
Then the j-index corresponding to the minimum $\chi$ is the nearest neighbor and so on, but I'm not satisfied by this method, since I feel I'm comparing each coefficient alone, rather than comparing the observation vector (as a whole) to all theoretical vectors.
Is there a better way to do this? It might be that chi-squared is sufficient (explanation below), but I'm no expert in this field (neither in statistics). And if a so, is there a way to estimate the error or uncertainty due to this comparison (or fitting should i say)
The data in $O$ and $T$ are decimals (positive and negative). But what I'm worried about is actually the range of the theoretical data. Let me explain more:
$T=\begin{matrix} 
t_{1,1} & t_{1,2} & \ldots  & t_{1,12}\\
t_{2,1} & t_{2,2} & \ldots & t_{2,12}\\
\vdots & \vdots & \ddots & \vdots \\
t_{n,1} & t_{n,2} & \ldots & t_{n,12}
\end{matrix} \ \ \ \ \ \ $ where $n=60000$, and   $O=[o_1,o_2,...,o_{12}]$
The minimum and maximum of each column vector of $T$ differs compared to another, such that the range of a column vector decreases as we move from one to the next. Let's say the {min,max}={-20,40} for the first column, and {min,max}={-2,3} for the 12th. Now here comes the problem (as I see it): a difference of 0.5 between $o_{1}$ and a value in the first column vector of $T$ isn't "weighted" equally as a difference of 0.5 between $o_{12}$ and a value in the 12th column vector of $T$. This is what I think the euclidean distance misses. I hope I made it clear enough.
Note: I'm new here, and sorry for any mistakes if I did any. 
 A: Now that you've expanded your description of the problem, I can suggest the Mahalanobis distance: letting $S$ be the covariance matrix of your dataset $T$, the distance is defined by
$$
d(o, t_j) = \sqrt{(o - t_j) S^{-1} (o - t_j)}
.$$
If the different attributes (1 through 12) are uncorrelated, then the covariance matrix will be diagonal and this corresponds just to standardizing the features by their relative spread (as measured by the standard deviation). In that case, the distance becomes
$$
d(o, t_j) = \sqrt{ \sum_{i=1}^{12}  \frac{(o_i - t_{ji})^2}{s_i^2} }
.$$
This standardized distance is also a very reasonable and typical thing to do on its own. The difference between the two is basically that the full Mahalanobis accounts for correlations between the components as well. If it's the case that, say, the 1st and 2nd components are usually very similar to one another, then the full Mahalanobis won't "double-count" that distance, whereas the standardized one will. Either choice can be reasonable depending on what your goals are for picking the "closest" points.
