I have conducted a series of pairwise relatedness estimations. The resulting relatedness values can technically range from 0 to 1. I have then clustered the individual estimates into groups/populations of interest, and then calculated an average relatedness value for each group. My supervisor then advised me to calculate a coefficient of variation of the mean, or standard error of the mean expressed as a percentage of the mean within groups (since sample size per group varies). This is supposed to give some measure of reliability of the averaged relatedness measure of the group.

Now, looking at the available literature, most papers using similar methods do not provide any measurement of this sort. They usually don't investigate the dispersion of their estimates within subgroups at all.

I noticed that when the mean of estimates within a group is close to zero, the CVmean becomes very high, and I read that the "standard" CV approaches infinity as the mean comes close to zero. This, in my opinion, would make its application for our purpose relatively useless, since a mean relatedness of 0 would be quite informative and not unexpected.

Would anyone be aware of an alternative and less problematic way to express the dispersion/variance of these estimates from the mean within a group? Each individual estimate has 95% confidence intervals associated with it, could there be a way to utilize those in some way?

I found one similar question approaching topic, but with no answer provided.

Edit to clarify: The estimates are relatedness values between individuals within a group. We want to know how closely related individuals within any group are, on average, and then use this information as a variable in other investigations. We essentially have, for example, pairwise estimates between ten individuals in one group, and would want to express how close the estimates are to the averaged estimated relatedness within that group. The spread from the average would hopefully tell us whether the average a good representation of the within-group relatedness, so we know we can use it. If lets say the relatedness values we get within a group are everywhere between -0.2 and 0.7, and the average is 0.1, the CVmean could potentially tell us if 0.1 actually does reflect the average relatedness within the group. Estimate here refers to an estimation of relatedness between two individuals from a population, based on comparisons of their genetic information.


1 Answer 1


a coefficient of variation ... is supposed to give some measure of reliability of the averaged relatedness measure of the group.

most papers using similar methods do not provide any measurement of this sort. They usually don't investigate the dispersion of their estimates within subgroups at all.

There is a third way here: provide a measure of dispersion within groups (a very good idea), but don't divide by the mean values.

There is no need to divide a measure of dispersion by the mean value. In fact, for evaluating differences among groups, you want to do something like the opposite. To estimate the reliability of an observed difference between groups, which presumably is of some interest, you start by dividing the observed difference by a pooled within-group error estimate. That's the basis of analysis of variance and linear regression.

Furthermore, in those contexts the error magnitude is assumed to be constant across the range of values. In that case, dividing the error by the mean imposes unnecessary systematic differences as mean values change.

The coefficient of variation, the ratio of standard deviation to the mean, can be informative if there's reason to think that errors are proportional to mean values. That certainly doesn't seem to be the case for your data. Furthermore, if errors are proportional to mean values you probably should be using a type of model that explicitly takes the error-mean relationships into account, like generalized linear models.

Just report a measure of within-group dispersion, and possibly use that to help evaluate the differences among the groups.

  • $\begingroup$ I might misread this, my apologies in this case, but I think you could have misinterpreted what this question was about. I edited it to clarify, hopefully. I suppose the CVmean was chosen, since it does take into account how many estimates there are within any group, and since what we are interested in is the mean value of estimates within a group, we do not compare between groups. If all the estimates within a group are close to the mean, the CVmean should reflect that quite well. The issue is how it does behave when the mean approaches 0. $\endgroup$
    – Leovar
    Commented Jun 20, 2021 at 0:39
  • $\begingroup$ @Leovar it’s the same whether or not you’re comparing among groups. The within-group dispersion should be fine on its own. Dividing by the mean only makes sense if that dispersion is approximately proportional to the mean. That doesn’t seem to be the case here, and it leads exactly to the problem you note: the result is unstable as the mean gets close to 0. $\endgroup$
    – EdM
    Commented Jun 20, 2021 at 3:29
  • $\begingroup$ I see, I think I know what you are aiming at, within-group dispersion is certainly not proportional to its mean. Is there any particular measure of dispersion you could recommend that you like working with? $\endgroup$
    – Leovar
    Commented Jun 20, 2021 at 8:41
  • 1
    $\begingroup$ @Leovar standard deviation is usually a good choice, unless there are outliers. In that case, the median absolute deviation can be better, as it’s insensitive to the exact values of the most extreme half of values. $\endgroup$
    – EdM
    Commented Jun 20, 2021 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.