Visualize Covariance when only probability mass and marginal functions are given

I am trying to intuitively understand Covariance like here. So if a random sample set given, I could draw rectangles with them, one of the cornes being fixated on mean $$(\overline{x},\overline{y})$$. Something like below.

I saw an example for Covariance where just the joint pmf was given and Covariance was calculated as below.

This was weird, because if I try to visualize the points location as per formula used above, it would be perfect rectangle like below, suggesting the X and Y are independent. It does not make sense to directly draw them from joint pmf, to observe any trend, then how come they could be used in teh Covariance formula? How do I visualize this particular situation when only pmf is given? I understand inequal probability would have had given a scatter if samples are drawn from this distribution, but then we should have used sample data in Covariance formula, not these right?

The above example was taken from Probability and Statistics for engineering and the sciences by Devore link here, page 208.

Update 1:
I have constructed 3d bar as below, to illustrate that probability also adding weight (here height), to the measure, which would not be evident from 2D perspective (which is inclusive in this case). Looking at 3D, one could say, oh ok green is more (with higher bars, larger volume), so positive covariance. Will this visual inference would be correct? If this is so, only issue here is the 3d bars are not rightly rendered issue created

• In your example, the random variables are discrete, which explains your difficulty in reproducing the first graph; I don't think that you can visualise the covariance between your particular set of random variables using the methodology in that link. – InfProbSciX Nov 3 '18 at 19:49
• (1) The six points in your second graph have different probabilities, which makes it difficult to visualize the covariance. (2) There are infinitely many joint distributions that have the marginal distributions of your $X$ and $Y.$ Only the one in which $p(x,y) = p_{{}_X}(x)\times p_{{}_Y}(y),$ for $x=100, 250$ and $y = 0, 100, 200$ has $X$ and $Y$ independent. For example, the upper left probability in your first table would be $p(0,100) = (.5)(.25) = 0.125$ (not $.20)$ and so on. – BruceET Nov 3 '18 at 21:15
• let us drop independence for a moment and focus on how to visualize them? Because I am writing a detailed tutorial for myself where I started with intuition of contrivance (and not yet gotten into correlation), so want my transition to be smooth. If visualization is for random sample and I am lost in visualizing the given example it would be a discontinuity in the flow of understanding. Instead of constructing rectangle, probably I should go for 3d cubes with height equal to respective joint probability? – Parthiban Rajendran Nov 4 '18 at 4:24

In an Answer to this question @Glen_b shows how to generate a bivariate normal sample with a given population correlation $$\rho.$$ Below, I use this method to show samples of size $$n = 2000$$ from each of three bivariate normal distributions with respective correlations $$\rho = .1, -.5, .9.$$ In each case $$\mu_x = \mu_y = 0$$ and $$\sigma_x = \sigma_y = 1,$$ so that all marginal distributions are standard normal.

set.seed(1103);  n=2000
rho = .1; x1 = rnorm(n);  z = rnorm(n);  y1 = rho*x1 + sqrt(1-rho^2)*z
rho =-.5; x2 = rnorm(n);  z = rnorm(n);  y2 = rho*x2 + sqrt(1-rho^2)*z
rho = .9; x3 = rnorm(n);  z = rnorm(n);  y3 = rho*x3 + sqrt(1-rho^2)*z
r1=round(cor(x1,y1),3); r2=round(cor(x2,y2),3); r3=round(cor(x3,y3),3)
par(mfrow=c(1,3))
plot(x1,y1, pch=".", main=paste("r = ",r1))
abline(h=0, col="green2"); abline(v=0, col="green2")
plot(x2,y2, pch=".", main=paste("r = ",r2))
abline(h=0, col="green2"); abline(v=0, col="green2")
plot(x3,y3, pch=".", main=paste("r = ",r3))
abline(h=0, col="green2"); abline(v=0, col="green2")
par(mfrow=c(1,1))


You are correct that concentrations of points in the first and third quadrants add positive terms to the sample covariance and hence tend to make the sample correlation positive; similarly, points in quadrants two and four add negative terms to the sample covariance.

However, do not confuse zero correlation with independence: if $$X$$ and $$Y$$ are independent, then $$\rho_{x,y} \approx 0.$$ But correlation near $$0$$ is not necessarily an indication of independence. (Perhaps see the Wikipedia article on correlation and dependence for examples.)

• thank you very much for detailed answer, but as I said I am in a flow from covariance and not yet into correlation. I have already visualized like above using random samples to illustrate the concept of covariance, only thing is I want such visualization for example also, else it would be a discontinuity – Parthiban Rajendran Nov 4 '18 at 4:25
• The correlation is the covariance divided by the product of the two standard deviations. I have no idea what you mean by 'discontinuity'. If you are looking for something that will make your method work, then perhaps take the comment by @InfProbSciX more seriously. – BruceET Nov 4 '18 at 5:15
• sorry, by discontinuity I meant as a metaphor, in my understanding. for discrete X and Y, for random samples we are able to visualize how co variance acts as a measure by plotting rectangles with one corner fixated at mean. But when a joint pmf is given, I am lost. The x and y from pmf are not samples per se, but all possible x and y values with different probabilities. Because they are not randomly scattered, which could have given a visual hint of covariance, they are distributed in rectangles though with diff probabilities. Thatis, when X=100, y could take multiple values. (tbc) – Parthiban Rajendran Nov 4 '18 at 5:20
• how could I then visualize them without taking in to account their diff probability is naturally next question because that is how I visualized for random samples earlier. What is the missing piece that is prohibiting me to visualize? Probably if they had given samples instead of pmf, I could have visualzed? If so, then covariance calculation for given pmf will not make any sense. If it makes sense, then there should be a way to visualize them just like we did for random samples. Isnt it? – Parthiban Rajendran Nov 4 '18 at 5:23
• The example in Jay Devore's book is a fine start. But I sense that you need to be familiar with a much wider range of examples for a more comprehensive understanding. The examples I gave and the ones in the Wikipedia link were my effort to help with that. – BruceET Nov 4 '18 at 5:31