Detecting Clusters of "similar" source codes Assume I have 400 students (that's in a big university) that have to do a computer science project, and that they have to work alone (no group of students). An example of project could be let "implementing a fast fourier transform algorithm in fortran"  (I know, that doesn't sound sexy but that makes my question simpler). I am the correcter and I want to sent routines to check if there are groups of student that have proposed implementation that are "too similar to be truly independently written". 
This is unsupervised search for clusters. I think the question is more about which attributes to use rather than the which clustering algorithm to use. The first thing I would do is a letter by letter histogram. Ideally, since cheaters are smarter than that, I would eventually try well chosen random permutations of letters to see if a good matching of letter's histogram (with permutation) exists. Also that those not explore the structure of the code, only the marginal distribution of letters... what solution do you have ? are there existing software or packages dedicated to that problem ? (actually in my old days computer science teachers claimed they had that type of tool, but I now suspect that they had something very simple)
I guess lawyer from software developments have that type of issues also (not with 1000 students, but with 2 large codes... which makes things harder) ? 
 A: The obvious pre-processing step is to merge files that are truly identical.
After that the key is normalization. At some point, students will start refactoring the code, renaming variables and such. Or reword the comments. A letter histogram is too much affected by this (plus it will capture a lot of the language properties).
A common technique is to use a language-specific parser and transform the source code into an abstract syntax tree. Then extract features from this. And maybe analyze the comments separately in parallel.
Then there's the line-based "longest common subsequence" approach. If you have a reasonably good similarity on single lines, you can search for the longest common subsequence of any two files. This will also yield a number of matches.
A: From the anti plagiarism world, I previously came across the notion of "Graph Isomorphism". Maybe you can take a look at that too.
LCS - Longest Common Subsequence is possible too. But try to compare all these solutions and see what's the best :)
