algorithm for finding similar items I have a group of items S, say there are 100 items in S, and I know some features of these items, i.e., color, size, ...Now, I have another group of items P, say there are 10000 items in P. What can I do to find a sub group of items in P that are similar to items in S? I know the corresponding features of all items in P, i.e., color, size... My original thinking is that I can use K nearest neighbor. But it seems that as KNN uses a majority vote, in order to use KNN, at least two different labels are in need in training data set while in my case I only have one label - all items in S would have the same label. Any ideas on how to find items in P that are similar to S??
 A: What about a very basic approach like this:
1) Define a distance metric between two items.
2) Find a distance between each item in P and S (you can create a representative item of S, or compare item in P to each item in S, and then sum the scores)
3) Take items in P that have the smallest distance. 
A: What about canonical correlation analysis (CCA)? CCA takes two sets of variables, one called the DVs and one called the IVs. It then identifies two canonical variables/variates, one for each set, which correlate at maximum. I like to think of these canonical variables as factors similar to FA/PCA, although the analogy does not strictly hold. You can then identify specific IVs and DVs which "load" high on their respective canonical variable. Depending on the canonical correlation and on the weights of the individual variables, you have identified two similar sets of variables. Of course, a second (third, ...) pair of canonical variables can be found up to $min\{k_{IV},k_{DV}\}$. Note that CCA reduces to simple regression if you have only one DV. I think pretty much every statistical software has it, but I don't know about the performance of CCA with 1000s of variables.
