I'm using the MATLAB code released by Eric P. Xing, related to their NIPS 2002 paper (pdf): "Distance metric learning, with application to clustering with side-information. Eric P. Xing, Andrew Y. Ng, Michael I. Jordan and Stuart Russell".
The code is available for download (.tar.gz) at this webpage.
When using the Newton-Rhapson method (Newton.m file), it is supposed to return a diagonal matrix, and when using the projections method (opt_sphere.m file), it is supposed to return a full matrix with entries greater than or equal to zero. Please see the paper for more on this.
However, when I try this on a sample dataset (Iris dataset), sometimes I get a matrix with negative entries when using the latter method. Similarly, with the former method, I sometimes get a matrix with two zero diagonals (that results in the transformed features being collapsed to a point).
Has anyone else experienced this before? Do you know what I am doing wrong?
As an example, consider the following code snippet (I have extracted the matlab code into the directory "code_metric_online"; these pairs of rows have the same labels, hence are similar: 30th and 42nd, 78th and 83rd, 9th and 49th; these pairs of rows have different labels, hence are dissimilar: 23rd and 61st, 96th and 150th, 45th and 80th):
addpath('code_metric_online/'); clear; load fisheriris; [N,d] = size(meas); data = meas; S = sparse(N, N); D = sparse(N, N); %S(9,49) = 1; S(30,42) = 1; S(78,83) = 1; %D(45,80) = 1; D(23,61) = 1; D(96,150) = 1; A = Newton(meas, S, D, 1); A %A = opt_sphere(meas, S, D, 100); %A transformed_data = data * (A^1/2)'; figure; scatter(transformed_data(:, 1), transformed_data(:, 2));
The resulting matrix
A in the above example will have two diagonal entries equal to zero, resulting in the plot being a single point. Similarly, if you comment out the Newton method and use opt_sphere instead, you will get a matrix
A with negative elements.
If however, you add two new constraints (by un-commenting
S(9,49) = 1; and
D(45,80) = 1;, then the plot will be a straight line.
I cannot understand this strange behavior, while in the paper, it is clearly said that A is greater than or equal to zero.