We are collecting certain metrics using (Graphite + Grafana) use them as a tool to monitor system health and performance.
For one of the latency metric, we get the total time as well as the latencies for all the sub-components it is composed of.
We display 99th percentile for all the values. However, if we sum up the 99th percentiles for latencies of sub-components, they do not equate to the 99th percentile of the total time.
Essentially it comes down if the percentiles can follow summation rules. i.e.
if
a + b + c + d = s
then,
p99(a) + p99(b) + p99(c) + p99(d) = p99(s) ?
Will this hold?