I'm using the Mann-Kendall function of the Kendall package in R to compute the statistics of the Mann-Kendall trend test of a huge time-series (19 millions elements). It has been running for 22 hours and it hasn't yet finished. Can you suggest a faster approach?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
I'd suggest thinking more about what you want to know about your data. The Mann-Kendall test is almost sure to be significant; with that many data points, the variance of Kendall's tau (the nonparametric correlation used here) is 2.33e-08, so a correlation of 0.001, which is unlikely to be practically significant, would still have a p-value of about 6e-11.
Computationally, the Mann-Kendall function is using Fortran under the hood, so it's unlikely it could be sped up; the problem is that there are 1.805e14 pairs to consider -- that's a lot of pairs!
answered Dec 13, 2011 at 14:47
-
-
6$\begingroup$ No, you should stop it and do something else. $\endgroup$ Commented Dec 14, 2011 at 17:20