Mutual information, considering each dimension as a random variable, thus each matrix as a set of pairs of numbers, should help in all cases, except for C, where I am not sure of the result.
See the discussion around Fig 8 (starting in p24) on regression performance analysis in the TMVA manual or the corresponding arxiv entry.