3
$\begingroup$

It is straightforward to verify that for two random variables $X$ and $Y$ with variances $\sigma^2_X \neq \sigma^2_Y$, we have that

$$\Big|{\rm Cov}(X, Y)\Big| \leq \max\{\sigma^2_X,\, \sigma^2_Y\}$$

On the other hand, is is not true in general that $\Big|{\rm Cov}(X, Y)\Big| \leq \min\{\sigma^2_X,\, \sigma^2_Y\}$

Assume $\sigma^2_X$ is the smaller variance. Then if we have

$$\sigma^2_X \leq \Big|{\rm Cov}(X, Y)\Big| \leq \sigma^2_Y \Rightarrow \sigma^2_X \leq |\rho| \sigma_X \sigma_Y \Rightarrow \frac {\sigma_X}{\sigma_Y}\leq |\rho|$$

Theoretically, this is perfectly feasible, the bi-variate normal case being the easiest example. But somehow it doesn't feel very "likely in practice", when $X$ and $Y$ are measured in the same units, to observe the ratio of standard deviations being smaller than the (absolute) correlation coefficient -but I may be wrong.

So I am looking for any kind of real-world-data-sets experience on such a phenomenon, in an attempt to informally assess whether it can be considered "likely to observe in practice" or not.

This relates to research in the following way: I am planning to use an approximating function which is piecewise, taking different functional forms depending on whether the argument of the approximated function is negative or not (which in turns depends on the above relations). And I need to obtain theoretical results prior to any empirical implementation, so it is not just a matter of "letting the data speak". Moreover this is not tied to any specific "part of the real world". In the interests of economy, it would be good to do all the theoretical work based on one hypothesis rather than derive all different scenarios from the beginning. And then it would be better if this hypothesis could be said to be the one "expected to hold" (informally).

$\endgroup$
  • 3
    $\begingroup$ I am voting to close this question on the grounds that the answers will be primarily opinion-based, though "too broad" or "unclear what you are asking" are also applicable grounds. Indeed, how will the OP use any answers? "3 out of 4 responders on stats.SE said that 5 out of 6 real-world data sets that they had occasion to examine had this property and so I expect that the data set that we are working on also enjoys this property." ?? $\endgroup$ – Dilip Sarwate Jan 25 '15 at 14:50
  • 3
    $\begingroup$ @DilipSarwate How can communicating specific experience be not opinion-based? As to "how the OP will benefit from the answers?", he will gather a first instance of evidence on whether a theoretically possible relation actually manifests in practice. What's wrong or unscientific with that? $\endgroup$ – Alecos Papadopoulos Jan 25 '15 at 16:05
  • 1
    $\begingroup$ Alecos, you are asking if someone has seen any actual data sets where $$\Big|{\rm Cov}(X, Y)\Big| \leq \min\{\sigma^2_X,\, \sigma^2_Y\}\tag{1}$$ It is entirely possible that practicing consulting statisticians (to which set I do not belong) have not encountered data sets of this type because if $X$ and $Y$ are uncorrelated (or have small correlation), then the experimentalists simply went on try something else and did not even bother to consult a statistician to see if something publishable could be salvaged from the data. (continued) $\endgroup$ – Dilip Sarwate Jan 25 '15 at 16:20
  • 1
    $\begingroup$ (Continuation) It is also possible that practicing consulting statisticians have seen many data sets of this type because what the experimentalists were trying to prove is that $X$ and $Y$ are uncorrelated, and now that they have data to support this hypothesis, they are consulting the statistician to publish this data in a manner that will withstand refutation. Either way, I fail to see how the information you are trying to collect is useful. 3 out of 4 dentists do recommend sugarless chewing gum, don't they? How does knowing this help? $\endgroup$ – Dilip Sarwate Jan 25 '15 at 16:24
  • 1
    $\begingroup$ @DilipSarwate Since you are not a practicing statistician, why are you trying a priori to exclude the possibility that they have something to say on the matter? The fact that you think that they won't have to say something, or something useful, just tells us that you wouldn't ask this question -and you didn't, I did. $\endgroup$ – Alecos Papadopoulos Jan 25 '15 at 17:14
5
$\begingroup$

Unless I've made some mistake (which I may have, I'm not clear-headed right now):

$\text{Cov}(X,Y)/\text{Var}(X) = \rho \sigma_y/\sigma_x$ can be made smaller or larger by choosing different units for $x$ or $y$ (e.g. going from dollars to cents or meters to millimeters or vice-versa).

As a result, I think you can do it simply by changing units.

$\endgroup$
  • 1
    $\begingroup$ That's kind of an important restriction, wouldn't you think? It should probably appear in your question. $\endgroup$ – Glen_b Jan 25 '15 at 2:08
  • 2
    $\begingroup$ Even in the same units, if for some reason $\sigma_x$ is much smaller than $\sigma_y$, that ratio might be large. $\endgroup$ – Glen_b Jan 25 '15 at 2:17
  • 2
    $\begingroup$ Personal anecdotes don't necessarily generalize to your specific unstated circumstances. I wouldn't have any basis on which to guess what the ratio of $\sigma$'s would be in your application, let alone the $\rho$. $\endgroup$ – Glen_b Jan 25 '15 at 2:22
  • 2
    $\begingroup$ I wish I could be more help on this. I don't know on what basis one could generalize to "likely in practice". The question as it stands now seems to be both impossibly broad, and relying essentially on personal opinion. What goes in the denominator of some probability assessment of 'likely'? $\endgroup$ – Glen_b Jan 25 '15 at 2:31
  • 3
    $\begingroup$ I am with Glen_b on this. In general, covariance and variance don't have the same dimensions or units of measurement. So your inequality is at worst meaningless and at best entirely dependent on conventions about choice of unit. I can't see a genuine problem here. $\endgroup$ – Nick Cox Jan 25 '15 at 11:46
4
$\begingroup$

I am not quite sure what the question is asking.

The absolute value of $\operatorname{cov}(X,Y)$, the covariance of $X$ and $Y$ is no larger than $\sigma_X\sigma_Y$ which is the geometric mean of the variances of $X$ and $Y$. Since $$\min\{\sigma^2_X,\, \sigma^2_Y\} \leq \sigma_X\sigma_Y \leq \max\{\sigma^2_X,\, \sigma^2_Y\},$$ it is certainly possible for the covariance to exceed $\min\{\sigma^2_X,\, \sigma^2_Y\}$. In other words, the question

Is it not true in general that $\big|\operatorname{cov}(X, Y)\big| \leq \min\{\sigma^2_X,\, \sigma^2_Y\}$?

has the answer that the desired relationship is not always feasible.

Consider, for example, $X$ and $Y$ having variances $6^2$ and $8^2$ respectively.

  • Suppose that the correlation coefficient $\rho$ is $\displaystyle \frac 56$.
    Then, $\displaystyle\operatorname{cov}(X, Y) = \rho\sigma_X\sigma_Y = \frac 56 \times 6 \times 8 = 40 > \min \{\sigma^2_X,\, \sigma^2_Y\} = 36,$ and $\displaystyle \frac{\sigma_X}{\sigma_Y} = \frac{6}{8} < \rho = \frac 56.$

  • With a smaller correlation coefficient $\displaystyle \frac 34$, we have that the covariance is $36$, same as $\sigma_X^2$, and of course $\displaystyle \frac{\sigma_X}{\sigma_Y} = \frac{6}{8} = \rho$.

  • If $\rho$ were even smaller, say $\displaystyle\rho = \frac 12$, the covariance is $24$ which is smaller than $\sigma_X^2$.

So it would appear that the OP is asking whether most real-life data sets that people have encountered (with $\sigma_X < \sigma_Y$) happen to have correlation coefficients that do not exceed $\displaystyle\frac{\sigma_X}{\sigma_Y}$ in magnitude.

$\endgroup$
  • $\begingroup$ In the middle of my question I write (the phrase in bold letters) that I am looking for "real-world data sets experience" relate to the issue. I thought it would be clear that this would be understood to mean "is this theoretical possibility actually observed?", but apparently it was not... Thanks for bringing in the geometric mean aspect. $\endgroup$ – Alecos Papadopoulos Jan 25 '15 at 10:40
4
$\begingroup$

The trees data from the R datasets package:

This data set provides measurements of the girth, height and volume of timber in 31 felled black cherry trees. Note that girth is the diameter of the tree (in inches) measured at 4 ft 6 in above the ground.

The correlation between girth & height is $0.52$, while the ratio of the standard deviations of girth to height is only $\frac{3.14''}{76.4''}=0.04$.

It's rather common to measure correlations between measurements of large magnitude to measurements of small magnitude with roughly similar coefficients of variation, & there's no particular reason such correlations should be small.

Note that if girth were expressed as circumference rather than diameter the ratio of standard deviations would be 0.13; in fact some queer way of expressing the observations might always be found to raise the value of the ratio over that of the correlation. So the question's about not just what's likely to be observed, but what's likely to be written down.

$\endgroup$
  • 3
    $\begingroup$ To be honest I'm baffled as to why anyone might think it unusual. $\endgroup$ – Scortchi Jan 25 '15 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.