4
$\begingroup$

I used the Mann Kendall trend test to determine the significance of the trends of the following time series data (sheet 1) and obtained a significant trend at 10% significance level for the month of April (autocorrelated) however, the Sen slope I obtained was zero. Similar results were obtained for other time series as well with a significance level set at 5%. I did the computations on R (modifiedmk package and trend package) as well as in XLSTAT (see sheet 2). XLSTAT was unable to compute the confidence intervals for the Sen slope for all the monthly data (Jan-Dec). I posted this question on various forums and the only seemingly logical explanation I received was on ResearchGate. But I am still unconvinced as I have not seen such a result in any of the papers that I have read on this topic. Can somebody please explain what's wrong here?

$\endgroup$
1
  • $\begingroup$ Trask's answer on ResearchGate points towards the solution, but their example (of many small upward steps followed by one large downward one) is incorrect: the Sen slope would be nonzero, not zero. The point, though, is that because MK uses only the signs of the differences while Sen uses the actual numerical slopes, the test results can differ. The difference can be huge when the majority of slopes are zero: that is, when there are many tied values in the dataset. This suggests the value of supplying a summary (or graphic) of your data. $\endgroup$
    – whuber
    Dec 22, 2021 at 15:06

1 Answer 1

5
$\begingroup$

The Mann-Kendall test is based on the signs of the differences in values whereas the Sen's slope estimate is the median of all the two-point slopes.

Sen's slope can be viewed as a weighted median of the differences in values, because the slope between two points $(t_i,x_i)$ and $(t_j,x_j)$ (with $t_i\ne t_j$) with values $x_i$ and $x_j$ is the difference $x_j-x_i$ weighted (multiplied by) the reciprocal distance $1/(t_j-t_i).$ This combination of the loss of quantitative information (MK) and weighting (Sen) can cause the two estimates, as well as associated tests of them, to be completely different.

How might we produce an example? One way to produce a zero Sen slope is to assure that more than half the two-point slopes are zero. Here is one of the simplest possible illustrations. It begins with a short upward trending segment. Counterbalancing that is a long level segment at a lower value. (How long? This segment of eight points contributes $\binom{8}{2}=28$ zero slopes out of all $\binom{8+3}{2}=55$ slopes, making the (bare) majority of slopes equal to zero.)

Nevertheless, there are enough points for MK to conclude the slope is significantly negative. There are, after all, only three pairs of points exhibiting upward trends (the three pairs within the first three observations) while there are $3\times 8 = 24$ pairs exhibiting downward trends, for a net sign difference of $3-24 = -21$ (this is the MK statistic). Since this overwhelming majority of points are in the downwards direction, MK is justified in concluding a negative trend exists.

Figure

To see the Mann-Kendall statistic, visualize lines between all points:

Figure 2

Of the $24+28+3 = 55$ distinct pairs, $24$ move down (from left to right) and only $3$ move up. This difference is unlikely to occur by chance when the points are randomly resorted.

To see the Sen slope, redraw all these lines emanating from a common point and choose the middle:

Figure 3

This figure had to randomly alter ("jitter") the slopes to resolve the many zero slopes. More than half are exactly zero (level), placing their median (middle) value in the horizontal position.

This helps us appreciate the difference in the two methods: the Sen's slope does not account for the many large downward slopes. The Mann-Kendall statistic finds there are many more downslopes than upslopes, ignoring the horizontal slopes altogether.

Comment

One common use of these procedures is with environmental monitoring data, as popularized by Richard Gilbert in the 1980's. Their attractiveness lies in their applicability to left-censored data: that is, nondetects aka "less-than" values. In these tests, any value reported as "less than $x$" is treated as equal to another such value, but less than any value reported as $y$ with $y\ge x.$

A typical monitoring dataset in a condition that first got worse (environmental contamination), was rectified, and improved to the point of being unable to detect the contaminant, would look substantially like the figure (where the red dots are the nondetects and have been plotted at a constant value lower than all the gray quantified observations). Thus, this situation is not a mere mathematical pathology or artificial construction: these circumstances actually exist--and with sufficiently long monitoring after the cleanup, can be expected to exist.

$\endgroup$
9
  • $\begingroup$ This was a brilliant explanation so thank you for this. Just one more clarification. Out of the 64 data points in the problem time series, only 15 have non-zero values while remaining are zeroes so basically, a majority of points are zeroes. So can I assume that due to a lot of these zero values in the time series, there were a lot of tied values which were zeroes which resulted in a lot of differences being zeroes hence, the chances for the median value of slope to be zero increased? Can I consider that the results of MK test and Sen slope estimation are independent of each other? $\endgroup$
    – HarshUp
    Dec 22, 2021 at 17:08
  • $\begingroup$ Yes to the first (those many tied values cause the Sen slope to be zero); no to the second. Although the MK and Sen results can become apparently unrelated in some cases, they aren't truly "independent" in the statistical sense. In many cases they will tend to give similar indications of trend or lack thereof. $\endgroup$
    – whuber
    Dec 22, 2021 at 17:25
  • $\begingroup$ Here is the data and graph of the problem time series. As you may notice, there are a lot of zeroes in the series. $\endgroup$
    – HarshUp
    Dec 22, 2021 at 17:28
  • $\begingroup$ Right--that's exactly the kind of dataset I have described, isn't it? I find the MK results unconvincing, though, because they rely on independence of all the one-year rainfall increments, whereas rainfall is known generally to be fairly strongly correlated over 5 to 7 year periods. Any such positive serial correlation would make the MK p-values smaller--perhaps much smaller--than they should be. To my eye, there's no trend here. $\endgroup$
    – whuber
    Dec 22, 2021 at 17:32
  • $\begingroup$ I eliminated the autocorrelation effects in R using variance correction approach suggested by Yue and Wang (2004). The significant trend was obtained after removing autocorrelation. The issue, I thought, lied in data insufficiency. Although, I don't know if it can be called data insufficiency because the zero values are genuine observations and it can't be helped that they were part of the time series. Is it just a discrepancy that may be attributed to the disadvantages/weaknesses of these tests? $\endgroup$
    – HarshUp
    Dec 22, 2021 at 17:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.