I'm using the Mann-Kendall function of the Kendall package in R to compute the statistics of the Mann-Kendall trend test of a huge time-series (19 millions elements). It has been running for 22 hours and it hasn't yet finished. Can you suggest a faster approach?
Tell me more
×
Cross Validated is a question and answer site for
statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
|
|
||||
|
|
|
I'd suggest thinking more about what you want to know about your data. The Mann-Kendall test is almost sure to be significant; with that many data points, the variance of Kendall's tau (the nonparametric correlation used here) is 2.33e-08, so a correlation of 0.001, which is unlikely to be practically significant, would still have a p-value of about 6e-11. Computationally, the Mann-Kendall function is using Fortran under the hood, so it's unlikely it could be sped up; the problem is that there are 1.805e14 pairs to consider -- that's a lot of pairs! |
||||
