How should I approach to cluster short (one-three words) strings?

I have approx 450.000 strings in a csv file. It's part names, so the strings are normally based on one-three words. I need to somehow group them together in an automated way so I can work with these clusters instead of part by part.

Here are a couple of random examples from the CSV file see above linked:

CENTERING DEVIC, GEAR, PLUG, ADAPTER, ADAPTER, ADAPTER, APPARATUS PANEL A50, BRACKET, BRACKET, BRACKET MOUNT CS1000"P"HD, BRACKET MOUNT CS1000"P"HD, CABLE, SIGNAL CABLE, CABLE, SIGNAL CABLE, CAP, CAPSCREW, CARRIER, CHAIN, CHECK VALVE, CHUCK, CHUCK, CLEVIS, CONDENSER SET, COVER, CRAWLER SET 11-12TON, CYLINDER TUBE, DUST COLLECTOR, ELBOW, ELBOW, FILLER PIPE, FILTER ELEMENT, FLUID LEVEL, GAUGE, FOOT, FORGING, LARGE CAPACITY HUB, GPS-box D-rigg with cabin


Here is what I plan to do to attack the problem:

1. Use Levenshtein to compare the strings. Take the first item of the array of strings and compare it to all others in the array and calculate a score for this first item.
2. Lets say I only allow 10% difference between the first string and the second, third,.,.., and the rest strings in the array. I will remove the first item and the other items in the array that had only 10% difference or less compared with the first string.
3. Continue with second string in the array and do the same calculation over and over again until I have walked through the whole array.
4. Items that remains in the array, that doesn't fit (10% > diff), I will need to handle manual...

Is this a good approach? I have read a bit about hierarchical clustering, is this a better approach or can anyone guide me in another better direction that I suggested?

• What are you trying to ultimately accomplish? The best way to cluster something depends on what you want to do with the clusters. Sep 9, 2017 at 19:20
• @Kodiologist Ultimately I want to categorize the parts according to TARIC, but I believe it will be easier to do so if I first categorize the parts in clusters and then for each cluster I connect then to TARIC. circabc.europa.eu/faces/jsp/extension/wai/navigation/… Sep 9, 2017 at 20:48
• The link seems to be broken (it's probably specific to your user session). So does TARIC comprise some sort of list of parts, and you need to match up your part numbers to TARIC's identifiers on the basis of the part name? If so, there might be a method to get you there faster than by using clustering and then sorting the clusters by hand. Sep 9, 2017 at 21:06
• @Kodiologist here is an example of a chapter 8431* ec.europa.eu/taxation_customs/dds2/taric/… Sep 9, 2017 at 21:17
• Beware that e.g. "dog" and "fog" are highly similar for, e.g., Levenshtein distance. But completely unrelated for humans. Sep 10, 2017 at 8:34