Related: Average distance in distance matrix
I'm looking for some way to compare euclidean distance matrices. The matrices I need to compare will have constant number of rows but varying number of columns. My goal is to find to best or nearest match (pick the best matrix). Ideally there is a function that takes such a matrix as an argument and outputs a single value that I can sort, but that may be too much to hope for. I'm looking for something for robust than min or max distance. Can I do better than average distance?
Above stream, I have two data frames or tables. The first is constant, and the second is a sequence of subsets or filters based on a 'code' column. For each pair of tables, I'm computing the euclidean distance matrix (which will have various dimensions since at each iteration I'm dynamically finding all columns in common. Unfortunately the columns of the two master tables only overlap by around 25% of a few thousand). Feel free to suggest other approaches besides euclidean distance or distance matrices in general, if my approach is paradigmatically flawed.
Statistically, I'm worried about the implication of having different sets of columns for each comparison. This results from generation of dummy variables, the underlying representations of which vary. Consequently, for the (admittedly arbitrary) cases I've examined, most of the columns are entirely zero's, since there is little overlap between the factors that are turned into dummies.