Clark-Evans test results contradict to L function results

Question

I have a data set including ~300 points and I would like to see if the point pattern tends to be clustered or not. To do this, I first performed the Clark-Evans test in R:

subSet_p <- ppp(subSet[,1], subSet[,2], c(0, 0.4), c(0, 0.4))
# test
clarkevans.test(subSet_p, alternative='clustered', nsim=100)

The results show that the p value is large than 0.05 so that the point patterns are not considered as clustered:

Clark-Evans test
No edge correction
Monte Carlo test based on 100 simulations of CSR with fixed n

data:  subSet_p
R = 1.0125, p-value = 0.4059
alternative hypothesis: clustered (R < 1)  

However, when I further performed the L function to the same data set, the plot indicates a strong clustering pattern:

subSet_p <- ppp(subSet[,1], subSet[,2],  c(0, 0.4), c(0, 0.4))
L_func <- Lest(subSet_p, correction = 'isotropic')
L_func$iso <- L_func$iso - L_func$r
L_func$theo <- L_func$theo - L_func$r

colnames(L_func) <- c('r', 'CSR, confidence', 'Observations') 

L_func <- melt(L_func, id.vars = 'r')

L_envelope <- envelope(subSet_p, fun = Lest, rank=(0.05 * (999 + 1)))

colnames(L_envelope) <- c('r', 'Observations','CSR, confidence', 'low', 'high')
L_envelope$low <- L_envelope$low - L_envelope$r
L_envelope$high <- L_envelope$high - L_envelope$r

The plot:

How to explain this discrepancy? Thank you.

My data: https://livejohnshopkins-my.sharepoint.com/:x:/g/personal/hmi1_jh_edu/EWxAso_A9I1BrFOV-zf5p54B8Rp_7XFC5VUdveOJRiysDw?e=TbMiqC

Answer 1

As Ege Rubak has explained, there is no reason why different statistical tests should have to agree. If they always gave the same answer, they would be the same test.

Your plot of the adjusted L function suggests that the point pattern exhibits a hard core (i.e. no points come close together) at short distances, and clustering at slightly longer distances. The Clark-Evans test statistic is based on nearest-neighbour distances, and at short distances there is no evidence of clustering. Tests based on simulation envelopes respond to longer distances and give a different outcome.

The test based on the simulation envelope has been performed incorrectly. The envelope function (from the spatstat package) has no argument called rank. It appears you wanted to conduct a test using 999 simulations; the appropriate argument is nsim=999. The plot shown in your question is probably based on nsim=19 simulations. Also, the plot shows a pointwise simulation envelope but you interpreted the result as a global test (yes/no answer) so you probably need a global simulation envelope (set global=TRUE in the call to envelope). Note: the envelope limits are not confidence limits; they are significance bands.

Another explanation for the discrepancy is that the point pattern may be spatially inhomogeneous, in particular, it may have non-uniform intensity (spatially-varying density of points). Both the Clark-Evans test and the L function assume uniform intensity, and are affected adversely by spatial inhomogeneity. This should be checked first before checking for clustering.

There are numerous other problems with the code. You are using the spatstat package for most of the work (the functions ppp, clarkevans.test, Lest, envelope) but then you start manipulating the data structure internally, by hand, and using the reshape2 package (function melt). This is dangerous because you've changed some of the internal data without updating the rest. The tidyverse is completely unnecessary here. If you want a plot of L(r)-r against r, simply do

E <- envelope(subSet_p, Lest, nsim=999, global=TRUE)
plot(E, . - r ~ r)

The object E belongs to class "fv" and should be manipulated using functions that work on this class.

See the spatstat book for further information.

Answer 2

It is perfectly normal for two different statistical tests to behave differently on a specific data set. In this case the Clark Evans test simply doesn't have enough statistical power or is sensitive enough to reveal the non-Poisson nature of the data where as the L-function is.

To illustrate the problem: Imagine we had to decide whether a series of coin tosses was fair. You may base your decision on the fraction of heads whereas I base my decision on the number of switches between heads and tails. For a series of 20 tosses with this outcome:

HHHHHHHHHHTTTTTTTTTT

You see a perfect fraction of 50% heads and would say the coin toss is fair (fail to reject H0 of fair coin). On the other hand I would say that it doesn't switch between H and T enough and would reject H0 of fair coin. Both things can be statistically valid since we only care about controlling how often we reject H0 when it is actually true (significance level).