I used the Python manifold library for multi-dimensional scaling on my distance matrix. Can I use k-means or k-nearest neighbours on X_transformed afterwards?

from sklearn import manifold 
from sklearn.manifold import MDS
mds = MDS(n_components=2, dissimilarity='precomputed')
X_transformed = mds.fit_transform(distanceMatrix)

X_transformed will have 2 columns now. Can I do k-means or k-nn on it? Is this correct for k-nn:

# Find nearest neighbors by orderering by distance from "0"
neighbours = distanceMatrix[0].drop(0,axis=0)

k = 5
orderedNeighbors = neighbours.sort_values() # Sort. Nearest neighbor first.
nearestNeighbors = orderedNeighbors[0:k]    # Select the k first entries
print("My nearest neighbors:")

Firstly, MDS tries to preserve the distances between samples but it's not guaranteed. So, you should calculate the final distance matrix with the new dataset. You can perform any analysis that you want with the transformed data, keeping in mind that you now operate over the manifold that the data is assumed to lie on.

You just need to choose your objective to go on since KNN is a supervised classification/regression algorithm while k-means is an unsupervised clustering algorithm. If you just find the neighbors of each sample, that's not called KNN. And, in vanilla KNN, you don't drop the same sample (e.g. for k = 1, every point's (in the training set) closest neighbour is itself).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.