# Relation between mean probability and overdispersion

I have proportion data for N observations x P variables. That is to say for each variable, I have proportions that vary between observations. A proportion is the number of successes divided by the number of trials.

I am interested in quantifying the variation of the proportions between the observations for each variable, and compare proportion variations between the variables. If you are interested, my data are available at https://filesender.renater.fr/?s=download&token=d0fce6e9-99eb-b4d0-c097-77e50feb5ae4 (the file sums.csv contains the number of successes and the file tots.csv contains the number of trials; observations in row and variables in column). In my data, the average proportion strongly varies between variables.

I tried to solve this problem using a beta-binomial distribution for each variable, considering that the unobserved proportions between observations are not the same but instead follow a Beta distribution with 2 parameters: the proportion mean $$\mu$$, and the overdispersion $$\rho$$. I believed the overdispersion $$\rho$$ parameter would reflect the variation of proportions between the observations, so I estimated it for each variable using VGAM R package as follows:

library(VGAM)
bbP<-function(i){
fit <- vglm(cbind(sums[,i], tots[,i]-sums[,i]) ~ 1, betabinomialff, trace = F)
print(i)
return(Coef(fit))
}
bbPv=t(sapply(1:ncol(sums),bbP)) # my alpha and beta parameters
rho=1/(bbPv[,1]+bbPv[,2]+1) # rho computed from alpha and beta
mu=bbPv[,1]/(bbPv[,1]+bbPv[,2]) # mu computed from alpha and beta
plot(mu,rho,log="xy",xlab="Mu",ylab="Rho") # Figure 1
cor.test(log(mu),log(rho))


I found that the overdispersion $$\rho$$ increases with the average proportion $$\mu$$ for my data. My question is: is it expected? I thought these two parameters should be independent, but I found cor(log($$\mu$$),log($$\rho$$))=0.905648 with p-value < 2.2e-16. I am surprised by these results I cannot understand. In addition, it means I cannot compare the variations of proportions between my variables and rank them based on that, since the overdispersion here depends on the average proportion.

I would greatly appreciate any feedback. thx