1
$\begingroup$

Are there any advanced results established regarding the behavior of the Covariance of two random variables other than the bounds on the correlation and independence when it is zero etc. which are usually summarized in introductory notes on this topic such as at this link? http://www.stat.yale.edu/~pollard/Courses/241.fall97/Variance.pdf

Some Possibilities for Advanced Properties

1)

For example, whether it is a convex or concave function and so on and under what restrictions on the density or joint density functions etc.?

---- It seems to be both convex or concave since it is linear in the variables.

2)

Where do the maximum and the minimum occur etc.?

3)

Also, to get the joint density (for example, the bivariate normal) we need the correlation coefficient which is based on the covariance. And to get the covariance we need the joint density? Seems like a cyclical; which came first - the chicken or the egg problem?

Please let me know if anything is not clear or if this question is too trivial or incorrect in some sense etc.

Steps Tried to get Maximum / Minimum

$$ Cov\left(X,Y\right)=\int\int\left(t-\mu_{X}\right)\left(u-\mu_{Y}\right)f_{XY}\left(t,u\right)\:dt\:du $$ $$ =\int\int\left(t\:u\right)f_{XY}\left(t,u\right)\:dt\:du-\mu_{X}\:\mu_{Y} $$ Taking derivatives and First Order Conditions, $$ \frac{\partial Cov\left(X,Y\right)}{\partial\mu_{X}}=\int\int ut\left[\frac{\partial\left\{ f_{XY}\left(t,u\right)\right\} }{\partial\mu_{X}}\right]\:dt\:du-\mu_{Y} $$ $$ \Rightarrow\mu_{Y}=\int\int ut\left[\frac{\partial\left\{ f_{XY}\left(t,u\right)\right\} }{\partial\mu_{X}}\right]\:dt\:du $$ $$ \frac{\partial Cov\left(X,Y\right)}{\partial\mu_{Y}}=\int\int ut\left[\frac{\partial\left\{ f_{XY}\left(t,u\right)\right\} }{\partial\mu_{Y}}\right]\:dt\:du-\mu_{X} $$ $$ \Rightarrow\mu_{X}=\int\int ut\left[\frac{\partial\left\{ f_{XY}\left(t,u\right)\right\} }{\partial\mu_{Y}}\right]\:dt\:du $$

Can we take derivatives as above, assuming the densities are differentiable? Is another approach advisable? Can we simplify this further?

The only material I could find was this. (related but not the same) http://www.math.tu-dresden.de/sto/schmidt/dsvm/dsvm2003-4.pdf

Related Questions Joint Density and Covariance between Two Random Variables with the same Mean and Variance

https://math.stackexchange.com/questions/74677/covariance-of-increasing-functions/74681

$\endgroup$
1
  • 1
    $\begingroup$ Much of this seems to make little sense (and makes no sense at all mathematically) because it appears to be written as if covariances were functions of real parameters, but as set forth at the beginning of the post they are functions of bivariate random variables. Taking derivatives with respect to means appears particularly meaningless, because if you are considering a location family parameterized by the mean, then changing the mean will not change any covariances at all. This holds true even for variables that do not have densities. $\endgroup$
    – whuber
    Feb 27, 2016 at 23:39

2 Answers 2

9
$\begingroup$

The only useful bound I'm aware of is the Cauchy-Schwarz inequality which says that

$$ |\text{Cov}(X, Y)| \leq \sqrt{\text{Var}(X) \text{Var}(Y)} , $$

but in general covariances are unbounded. One way to see this is to observe that the Cauchy-Schwarz inequality is tight (consider when $X$ and $Y$ are perfectly correlated) and variances can be arbitrarily large.

$\endgroup$
2
$\begingroup$

This question is closely related to this previous question but also includes the incorrect belief that random variables are independent if they have zero correlation.

... on the correlation, and independence when it is zero etc.

I will answer only a simpler version of the OP's "chicken and egg" question:

to get the joint density (for example, the bivariate normal) we need the correlation coefficient which is based on the covariance. And to get the covariance we need the joint density? Seems like a cyclical; which came first - the chicken or the egg problem?

The simpler version relates to the variance of a normal random variable and asks

To find the density of a (zero-mean) normal random variable, we need to know the variance, and to get the variance we need the density. Seems like a cyclical; which came first - the chicken or the egg?

Answer:

If we are told that a zero-mean normal random variable $X$ has variance $\sigma^2$, then we can write down the density function as $$f(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^2}{2\sigma^2}}, ~-\infty < x < \infty. \tag{1}$$ It is not necessary to use the density to compute the variance since we know the value of the variance already, but if we do choose to proceed with the calculation -- preferably without asking on stats.SE or math.SE for help in calculating the value of $\operatorname{var}(X) = E[X^2] = \int_{-\infty}^\infty x^2f(x)\,\mathrm dx$ -- we come up with the result that the variance is indeed $\sigma^2$. So, in this case, the egg (value of the variance) hatched into the chicken (the density) and then the chicken laid the egg when we went through all the rigmarole of finding the variance. We can, if we like, say to the density (as Dorothy Parker did to a friend who had just had a baby): "Good job! We all knew you had it in you!"

On the other hand, if we simply told that a zero-mean random variable has density function $$f(x) = e^{-\pi x^2}, ~ -\infty < x < \infty,\tag{2}$$ and are asked to find its variance, we can proceed to try and calculate the variance of $X$ as in the previous paragraph, again hopefully without asking for help in computing the value of $\int_{-\infty}^\infty x^2e^{-\pi x^2}\,\mathrm dx$, or we can muse that the density function in $(2)$ looks a heck of a lot like the density function in $(1)$. The constant term $\frac{1}{\sigma\sqrt{2\pi}}$ in $(1)$ equals $1$ if $\sigma\sqrt{2\pi} = 1$, that is, if $\frac{1}{2\sigma^2} = \pi$, and by golly, that makes the exponent $-\frac{x^2}{2\sigma^2}$ equal $-\pi x^2$. So, $(2)$ is in fact the density of a zero-mean normal random variable whose variance is $\frac{1}{2\pi}$. At this time, we run wet and naked through campus streets crying "Eureka!" and burst into our professor's office to tell him that we have not just found the value of the variance as the professor had asked for but also discovered that the random variable in question is a normal random variable. So in this case, the chicken (density) came first and laid the egg of the variance. Knowing the value of the variance, we can, if we wish, write down the density in $(2)$ as $$f(x) = \frac{1}{\sqrt{\frac{1}{2\pi}}\sqrt{2\pi}} e^{-\frac{x^2}{2\left(\sqrt{\frac{1}{2\pi}}\right)^2}}, ~-\infty < x < \infty \tag{3}$$ which is a one-to-one match with the form written in $(1)$ so that everybody can tell immediately what the value of the variance is. In short, the egg hatches into the density stated exactly as one should write normal densities instead of the dubious form $(2)$ which is clearly designed to conceal rather than reveal.


All right, then. Suppose $X$ and $Y$ have joint density function $$f_{X,Y}(x,y) = \frac{1}{\pi}e^{-2x^2-2y^2+2\sqrt{3} xy},~ -\infty < x, y < \infty.$$ What are the means and variances of $X$ and $Y$ and what is their covariance? Which came first? the chicken or the egg?

Now let the down-votes begin....

$\endgroup$
2
  • $\begingroup$ @DilipSarawate ... Thanks for your detailed clarification. Before the down votes, let us be very clear that the question and this answer make explicit a very important point (subtle and unmentioned usually) that we assume a certain bi-variate density and from that everything else follows, variance, marginal densities, covariance etc.. But almost all online material shows the bivariate normal using the correlation coefficient; and then we all know to get the covariance (correlation) we need the joint density; hence this confusion can arise. $\endgroup$
    – texmex
    Feb 28, 2016 at 3:19
  • $\begingroup$ @DilipSarawate Also when you mention that the covariance can take on any value in the range (in the related question: stats.stackexchange.com/questions/198690/…). Is it correct to think about whether it can have a maximum or a minimum based on any parameters? I have tried above with the mean ... But please suggest if this is possible or if any other parameters can be considered. $\endgroup$
    – texmex
    Feb 28, 2016 at 3:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.