# Independence of Mean and Variance of Discrete Uniform Distributions

In the comments below a post of mine, Glen_b and I were discussing how discrete distributions necessarily have dependent mean and variance.

For a normal distribution it makes sense. If I tell you $$\bar{x}$$, you haven't a clue what $$s^2$$ is, and if I tell you $$s^2$$, you haven't a clue what $$\bar{x}$$ is. (Edited to address the sample statistics, not the population parameters.)

But then for a discrete uniform distribution, doesn't the same logic apply? If I estimate the center of the endpoints, I don't know the scale, and if I estimate the scale, I don't know the center.

What's going wrong with my thinking?

EDIT

I did jbowman's simulation. Then I hit it with the probability integral transform (I think) to examine the relationship without any influence from the marginal distributions (isolation of the copula).

Data.mean <- Data.var <- rep(NA,20000)
for (i in 1:20000){
Data <- sample(seq(1,10,1),100,replace=T)
Data.mean[i] <- mean(Data)
Data.var[i] <- var(Data)
}
par(mfrow=c(2,1))
plot(Data.mean,Data.var,main="Observations")
plot(ecdf(Data.mean)(Data.mean),ecdf(Data.var)(Data.var),main="'Copula'")


In the little image that appears in RStudio, the second plot looks like it has uniform coverage over the unit square, so independence. Upon zooming in, there are distinct vertical bands. I think this has to do with the discreteness and that I shouldn't read into it. I then tried it for a continuous uniform distribution on $$(0,10)$$.

Data.mean <- Data.var <- rep(NA,20000)
for (i in 1:20000){

Data <- runif(100,0,10)
Data.mean[i] <- mean(Data)
Data.var[i] <- var(Data)

}
par(mfrow=c(2,1))
plot(Data.mean,Data.var)
plot(ecdf(Data.mean)(Data.mean),ecdf(Data.var)(Data.var))


This one really does look like it has points distributed uniformly across the unit square, so I remain skeptical that $$\bar{x}$$ and $$s^2$$ are independent.

• That's an interesting approach you've taken there, I'll have to think about it. Aug 18 '19 at 17:12
• The dependence (necessarily) gets weaker at larger sample sizes so it's hard to see. Try smaller sample sizes, like n=5,6,7 and you'll see it more easily. Aug 19 '19 at 0:09
• @Glen_b You're right. There's a more obvious relationship when I shrink down the sample size. Even in the image I posted, there appears to be some clustering in the lower right and left corners, which is present in the plot for the smaller sample size. Two follow-ups. 1) Is the dependence necessarily getting weaker because the population parameters can be varied independent of each other? 2) It seems wrong that the statistics would have any kind of dependence, but they clearly do. What causes this?
– Dave
Aug 19 '19 at 0:30
• One way to get some insight is to examine the special features of the samples that get into those 'horns" at the top of Bruce's plots. In particular note that at n=5, you get the largest possible variance by all the points being close to 0 or 1, but because there's 5 observations, you need 3 at one end and 2 at the other, so the mean must be near to 0.4 or 0.6 but not near 0.5 (since putting one point in the middle will drop the variance a bit). If you had a heavy tailed distribution, both mean and variance would be most impacted by the most extreme observation ... ctd Aug 19 '19 at 0:40
• ctd... and in that situation you get a strong correlation between $|\bar{x}-\mu|$ and $s$ (giving two big "horns" either side of the population center on a plot of sd vs mean) -- with the uniform this correlation is somewhat negative. ... With large samples you'll head toward the asymptotic behavior of $(\bar{X},s^2_X)$ which ends up being jointly normal. Aug 19 '19 at 0:42

jbowman's Answer (+1) tells much of the story. Here is a little more.

(a) For data from a continuous uniform distribution, the sample mean and SD are uncorrelated, but not independent. The 'outlines' of the plot emphasize the dependence. Among continuous distributions, independence holds only for normal.

set.seed(1234)
m = 10^5; n = 5
x = runif(m*n);  DAT = matrix(x, nrow=m)
a = rowMeans(DAT)
s = apply(DAT, 1, sd)
plot(a,s, pch=".")


(b) Discrete uniform. Discreteness makes it possible to find a value $$a$$ of the mean and a value $$s$$ of the SD such that $$P(\bar X = a) > 0,\, P(S = s) > 0,$$ but $$P(\bar X = a, X = s) = 0.$$

set.seed(2019)
m = 20000;  n = 5;  x = sample(1:5, m*n, rep=T)
DAT = matrix(x, nrow=m)
a = rowMeans(DAT)
s = apply(DAT, 1, sd)
plot(a,s, pch=20)


(c) A rounded normal distribution is not normal. Discreteness causes dependence.

set.seed(1776)
m = 10^5; n = 5
x = round(rnorm(m*n, 10, 1));  DAT = matrix(x, nrow=m)
a = rowMeans(DAT);  s = apply(DAT, 1, sd)
plot(a,s, pch=20)


(d) Further to (a), using the distribution $$\mathsf{Beta}(.1,.1),$$ instead of $$\mathsf{Beta}(1,1) \equiv \mathsf{Unif}(0,1).$$ emphasizes the boundaries of the possible values of the sample mean and SD. We are 'squashing' a 5-dimensional hypercube onto 2-space. Images of some hyper-edges are clear. [Ref: The figure below is similar to Fig. 4.6 in Suess & Trumbo (2010), Intro to probability simulation and Gibbs sampling with R, Springer.]

set.seed(1066)
m = 10^5; n = 5
x = rbeta(m*n, .1, .1);  DAT = matrix(x, nrow=m)
a = rowMeans(DAT);  s = apply(DAT, 1, sd)
plot(a,s, pch=".")


• Use ecdf on your last one. The scatterplot is wild! Anyway, if a uniform variable has dependence between $\bar{x}$ and $s^2$, how is it that we're getting some information about one by knowing the other, given that we can stretch the range or shift the center willy nilly and not affect the other value? If we get $\bar{x}=0$, we shouldn't know if $s^2 = 1$ or $s^2=100$, similar to how we can stretch the normal distribution without affecting the mean.
– Dave
Aug 18 '19 at 19:49
• The criterion of independence is demanding. Lack of independence btw two RVs does not guarantee that it's easy to get info about one, knowing the value of the other. // In (d), not sure what ECDF of A or S would reveal. // Scatterplot in (d) shows 6 'points', images under transformation of 32 vertices of 5-d hypercube with multiplicities 1, 5, 10, 10, 5, 1 (from left to right). Multiplicities explain why 'top two' points are most distinct. Aug 18 '19 at 20:41
• I don't mean that it's easy to get info about one if you know the other, but if you have independence, all you can go by is the marginal distribution. Consider two standard normal variables $X$ and $Y$ with $\rho=0.9$. If you know that $x=1$, you don't know what $y$ equals, but you know that a value around $1$ is more likely than a value around $-1$. If $\rho=0$, then a value around $1$ is just as likely as a value around $-1$.
– Dave
Aug 18 '19 at 20:52
• But that's for a nearly-linear relationship btw two standard normals. Mean and SD of samples are not so easy. Aug 18 '19 at 20:59
• @Dave you do have information about one when you know the other. For example if the sample variance is really large, you know the sample mean isn't really close to 0.5 (see the gap at the top-centre of the first plot, for example) Aug 19 '19 at 0:12

It isn't that the mean and variance are dependent in the case of discrete distributions, it's that the sample mean and variance are dependent given the parameters of the distribution. The mean and variance themselves are fixed functions of the parameters of the distribution, and concepts such as "independence" don't apply to them. Consequently, you are asking the wrong hypothetical questions of yourself.

In the case of the discrete uniform distribution, plotting the results of 20,000 $$(\bar{x}, s^2)$$ pairs calculated from samples of 100 uniform $$(1, 2, \dots, 10)$$ variates results in:

which shows pretty clearly that they aren't independent; the higher values of $$s^2$$ are located disproportionately towards the center of the range of $$\bar{x}$$. (They are, however, uncorrelated; a simple symmetry argument should convince us of that.)

Of course, an example cannot prove Glen's conjecture in the post you linked to that no discrete distribution exists with independent sample means and variances!

• That's a good catch about statistic versus parameter. I've made a pretty extensive edit.
– Dave
Aug 18 '19 at 16:49