0
$\begingroup$

I am working on an an application that is part managed code and part unmanaged code. Because of the unmanaged code part, there was a requirement in the specifications that states that memory usage may not grow during the operation of the application. To that end, we measure the memory usage at a certain interval, for a long enough time.
Now, because the application is partly managed, it could very well be that you take the first measurement just after a garbage collection, and take the final measurement just before one. In that case, the requirement fails, even though there is nothing wrong...
So, the requirement was changed. Now it states that memory usage may not show a (growing) trend during operation. To quantify that requirement, we chose to compute the Kendall tau-b coefficient (as defined here) of the memory usage against the running time of the application.
At this stage of development, we are quite confident that there are no memory leaks in the application anymore. And we see that in the data points. Sometimes the memory grows a bit, sometimes it shrinks, but most of the time, the memory usage doesn't change at all. We are talking about 4 or 5 changes in about 100 data points.
Because Kendall tau-b accounts for ties, and most of our data is tied, the denominator of the tau-b formula becomes relatively small. That makes the few changes in the data become much more relevant, and the rank coefficient becomes quite large for data where basically nothing much happens... And indeed, the new requirement also fails a lot of the time.

My questions are these: Is Kendall Tau the correct approach to prove no trend in data with many ties? What other methods can I use?

$\endgroup$
1
$\begingroup$

Unless you calculate a weighted version of $\tau_b$, all your data influence the result equally. So this approach does not account for changes in trend. In other words: if the trend is initially flat and then starts to be positive, it will take you a longer time to pick it up.

If this is fine with you, I'd recommend a simple linear regression of memory usage against time (e.g., in seconds since system start). Then check whether the trend coefficient is significantly different from zero using a standard $t$ test.

Alternatively, especially if you anticipate changes in the trend, try fitting trended Exponential Smoothing to your memory usage, e.g., Holt's method. This gives you a "local trend" estimate. You can then monitor whether this smoothed trend component goes outside a specified control limit. One advantage is that this component has a direct interpretation as the increase or decrease per time interval.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.