Relation between covariance and joint distribution

Question

This is an extremely basic question in probability theory. Namely, for any two variables $x$ and $y$, if $\mathrm{cov}(x,y)$ is not 0 (in the population), what does that imply about their joint distribution? In other words, how is the joint distribution of variables related to their covariance?

Answer 1

I would ask the opposite question: what is the implications of zero covariance for any surviving dependence? The two variables can certainly be stochastically dependent even though their covariance is zero, but what kind of dependencies, and what kind of bivariate joint distributions, are excluded if covariance is zero?

Some examples:

a) Many-many "named" bivariate continuous joint distributions (i.e. joint distributions where the two marginals belong to the same family).

b) Bivariate continuous distributions of the Farlie-Gumbel-Morgenstern family

$$H_{X,Y}(x,y)=F_X(x)G_Y(y)\left(1+\alpha\big(1-F_X(x)\big)\big(1-F_Y(y)\big)\right), \;\; \alpha <1$$

for two random variables with arbitrary marginal distribution functions $F_X(x)$ and $G_y(y)$. Here $\alpha =0$ is necessary and sufficient for zero covariance and stochastic independence. So here you cannot have the one without the other.

c) Finally one must remember that although covariance is usually described as "reflecting the "linear" dependence among two variables", this may mislead because when a random variable $X$ is a pure non-linear function of another one $Y$, almost always their covariance will be non-zero. Consider a very simple case, let

$$X = Y^2 \implies \text{Cov}(X,Y) = E(XY) - E(X)E(Y) = E(Y^3) - E(Y^2)E(Y)$$

This in general won't be zero.

etc. Certainly, infinite ways to model stochastic dependence with zero covariance do exist, but the above shows that if one wants to go that way, "off-the-shelf" bivariate distributions won't do, nor will the postulation of a purely non-linear relationship.

This is why Copulas is perhaps the way to go, as a comment suggested since, from this perspective, they allow us a systematic way to model dependence with any marginal distributions. This is important, because when looking at data, we can more easily describe a distribution for each data series separately, and it is convenient here to call on our stock of well-known and studied marginal distribution families (and each variable may appear to have a marginal that belongs to a different family). Their joint distributions may be non-standard and so non already studied. Then we look at Copulas to describe the dependence.