0
$\begingroup$

I am trying to determine whether a group of objects ($n = 100$) are spatially ordered (on a plane) with respect to the value of a particular property $p$ (a scalar).

If I had a strong sense of which single direction this ordering might occur along, I could simply project the $x$ and $y$ positions onto this vector and correlate this projection with the property value. But I don't - and I'm not even sure if it would be a linear or even monotonic relationship.

So as a more general metric I've used correlation. First, I generated two new vectors: one which contains all of pairwise differences among the values of the $p$, and the other which contains the Euclidean distances between the pairs of objects. Then I calculate the correlation coefficient between the two vectors.

This metric seems to produce sensible results. Data sets which have (by construction) a topographic organization produce higher values of correlation than those which do not. My question is: what statistical test can I use to accurately determine whether the correlation value is "significant?" Or what is a more appropriate (but equally general) metric to use?

I am fairly sure that the standard p-value associated with the correlation coefficient is going to be totally invalid because the $n^*$ (length of the vectors) used to calculate $p$ is $n^2/2 - n$ (i.e. $>> n$). So I tried calculating $p$ using the actual value of $n$, instead of the inflated one, but that seems to underestimate the significance of a particular value of $r$, which I'm inferring from the distribution of p values on simulated null data. I wrote some code (Matlab) to illustrate:

N = 10;

n = 10000;

p = nan(n,1);
p2 = nan(n,1);
pCorr = nan(n,1);
pCorr2 = nan(n,1);

for i = 1:n

    x = randn(N,1);
    y = randn(N,1);

    dx = nan(N);
    dy = nan(N);

    for j = 1:N
        for k = 1:N

            if k >= i
                continue
            end

            dx(j,k) = abs(x(j) - x(k));
            dy(j,k) = abs(y(j) - y(k));

        end
    end

    [r,p(i)] = corr(dx(:),dy(:),'rows','complete');

    p2(i) = PvalPearson('b',r,N);

end

where PvalPearson is:

t = rho.*sqrt((n-2)./(1-rho.^2));
p = 2*tcdf(-abs(t),n-2);

There is obviously no relationship between x and y, and yet this procedure produces p < 0.01 26% of the time, and p2 about 0.1% of the time, instead of the expected 1%.

enter image description here

$\endgroup$
0
$\begingroup$

It looks like one route is a permutation test. Here is the code:

N = 10;

n = 5000;
nPerm = 100;

p = nan(n,1);

for i = 1:n

    x = randn(N,1);
    y = randn(N,1);

    rPerm = nan(nPerm,1);

    for ii = 1:nPerm

        if ii > 1
            yIdx = randperm(N);
            yPerm = y(yIdx);
        else
            yPerm = y;
        end

        dx = nan(N);
        dy = nan(N);

        for j = 1:N
            for k = 1:N

                if k >= i
                    continue
                end

                dx(j,k) = abs(x(j) - x(k));
                dy(j,k) = abs(yPerm(j) - yPerm(k));

            end
        end

        rPerm(ii) = corr(dx(:),dy(:),'rows','complete');

    end

    p(i) = sum(rPerm(2:end) > rPerm(1))/n;

end

Which gives a sensible distribution of p-values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.