Sign of Covariance and of Spearman's Rho

Question

Has anybody available a proof that the covariance between two variables has always the same sign as Spearman's Rho, assuming that both are not zero, or an explanation / counterexample to show why this is not the case?

I am talking about the "population" (theoretical) magnitudes, not their sample counterparts. Namely, for $X, Y$ two random variables with distribution functions $F_X, F_Y$, and with all needed moments, co-moments, etc, existing,

$$\text{Cov}(X,Y) = E(XY) - E(X)E(Y)$$ while

$$\rho_s(X,Y) = \text{Cov}[F_X(X),F_Y(Y)]$$

I know that if $X,Y$ are Quadrant Dependent ($QD$), positively or negatively, this indeed holds,

$$(X,Y) = QD \implies \text{sign}\left\{\text{Cov}(X,Y)\right\} = \text{sign}\left\{\rho_s(X,Y)\right\}$$

...again, if both are not zero. But what if $QD$ cannot be established or doesn't hold?

What I am eventually after is a proof that if $h(Y)$ is an increasing monotonic transformation of $Y$, then $\text{sign}\left\{\text{Cov}(X,Y)\right\} = \text{sign}\left\{\text{Cov}(X,h(Y))\right\}$. I know that this appears strongly intuitive and even "self-evident", but I could not find such a proof anywhere, neither did I manage to prove it myself. More precisely, what I want to show is that, if both are not zero, they cannot have opposite signs.

Now, since Spearman's Rho is invariant to monotonic transformations we do have $\rho_s(X,Y) = \rho_s(X,h(Y))$, so a way to prove the "same sign" result for the covariances, would be to prove that the covariance has always the same sign as Spearman's Rho, hence this question.

I have found an old beautiful expression for the covariance due to W. Hoeffding that brings the $\text{Cov}$ and $\rho_s$ definitions "very close", but I could not prove the general statement without assuming Quadrant Dependence.

Of course, if someone has something directly on the "same sign" (desired) result for the covariances, it would be equally helpful.

UPDATE
I found this question that is related but not identical. As already mentioned, it modifies my question as follows: "Assume that both measures are not zero. Can they have opposite signs?"

Answer 1

There are many counterexamples. But let's address the underlying question:

What I am eventually after is a proof that if $h$ is an increasing monotonic transformation, then $\operatorname{Sign}\{\operatorname{Cov}(X,Y)\}=\operatorname{Sign}\{\operatorname{Cov}(X,h(Y))\}$.

This is false.

The first counterexample is the discrete uniform distribution $F$ on the $(x_i,y_i)$ points $(1,8.1), (2,9.1), (3,10.1), (4,11.1), (5,12.1), (6,13.1), (7,0.1),$ here depicted by plotting those seven points as red circles in the left panel:

Consider the family of Box-Cox transformations

$$h_p(y) = \frac{y^p - 1}{p\, C} + 1$$

where the constant $C$ is chosen to make the values of $h_p(y_i)$ comparable to those of $y$ (for instance, by setting $C$ to be the $p-1$ power of the geometric mean of the $y_i$) and $1$ is added to make $h_1$ the identity. These are all monotonic; an example is shown for $p=2$ in the right panel. Their effects on the covariance are plotted in the middle panel. It shows a change from negative covariance (due to that outlying point at the bottom left) to positive covariance (because the transformation makes the point just a little less outlying, reducing its negative effect on the otherwise strong positive covariance of all the other data).

In particular, to be perfectly explicit, you may compute that

$$h(y_i,2) = (7.0, 8.6, 10.4, 12.4, 14.5, 16.8, 0.908),$$

giving $\operatorname{Cov}(x_i,y_i) = -7/3 \lt 0$ and $\operatorname{Cov}(x_i, h(y_i,2))=0.39217 \gt 0.$ The points $(x_i, h(y_i,2))$ are plotted as hollow blue triangles in the left panel.

The second counterexample is a continuous version of the first. Let $(U,V)$ have any continuous distribution supported on $[-1,1]\times[-1,1].$ For any real number $\epsilon$ define

$$(X_\epsilon, Y_\epsilon) = (X,Y) + \epsilon(U,V).$$

Provided $\epsilon\ne 0,$ $(X_\epsilon, Y_\epsilon)$ has a continuous distribution (see Is the sum of a continuous random variable and mixed random variable continuous?). Provided $|\epsilon| \lt 1/10,$ the support of $(X_\epsilon, Y_\epsilon)$ is in the first quadrant (strictly positive in both variables), implying the Box-Cox transformations can be applied to $Y_\epsilon.$ You can perform the calculations confirming that the covariance of $(X_\epsilon,Y_\epsilon)$ is a continuous function of $\epsilon.$ Ergo, for sufficiently small $\epsilon,$ the first counterexample shows the covariance of $(X_\epsilon,Y_\epsilon)$ is negative while that of $(X_\epsilon, h_2(Y_\epsilon))$ is positive, QED.

Answer 2

I say they can have opposite signs.

Let's look at the following simulation.

# Set a random seed so that everyone can get the same results
#     
set.seed(1)

# Import the library that simulates correlated bivariate data
#  
library(MASS) 

# Simulate bivariate normal data with standard normal 
# marginals and 0.9 Pearson correlation. To those 99 
# observations, add a gigantic outlier completely out 
# of the mainstream of the other 99 points. This is why 
# we end up with negative covariance.
#  
X <- rbind(mvrnorm(99,c(0,0),matrix(c(1,0.9,0.9,1),2,2)),c(-10000,10000)) 

# Plot the data
#  
plot(X[,1],X[,2]) 

# Calculate the covariance of the sample. When we regard 
# the simulated data as a discrete population, this is 
# the population covariance.
#  
cov(X[,1],X[,2]) # comes out negative, as the plot suggests

# Calculate the sample Spearman correlation, which is 
# positive, since 99% of the data follow an upward trend.
#  
cor(X[,1],X[,2],method='spearman') # comes out positive

However, we can take the simulated data as a discrete population.

# Apply the empirical CDF function to perform the probability
# integral transform. If we regard the sampled data as a
# discrete population, we have tricked R into calculating the
# population Spearman correlation.
#  
cov(ecdf(X[,1])(X[,1]),ecdf(X[,2])(X[,2])) # Positive, same value as before

The "ecdf" (empirical CDF) tricks R into making the population CDF of this discrete variable, so I think we're working at the population level and that this is a counterexample.

Answer 3

To enhance the value of this thread I will lay out why Quadrant Dependence implies that
a) Covariance will have the same sign as Spearman's Rho if both are not zero
b) The sign of covariance is not affected by strictly increasing monotonic transformations, if it remains non-zero.

I will show it for continuous distributions with densities, but this is not a critical condition.

Let $X$, $Y$ be two random variables with joint distribution function $F_{XY}(x,y)$, marginal distribution functions $F_X(x), F_Y(y)$ and marginal density/probability mass functions $f_X(x), f_Y(y)$. Then we have

\begin{cases} \text{Positive Quadrant Dependence iff} \;\;\; F_{XY}(x,y) - F_X(x)F_Y(y) \geq 0\;\;\; \forall (x,y)\\ \text{Negative Quadrant Dependence iff}\;\;\ F_{XY}(x,y) - F_X(x)F_Y(y) \leq 0\;\;\; \forall (x,y) \end{cases}

Note that the crucial condition is the "for all $(x,y)$" qualifier.

Now the "beautiful covariance formula of Hoeffding" is

$$\text{Cov}(X,Y) = \int\int_{S_{XY}}[F_{XY}(x,y) - F_X(x)F_Y(y)] dx dy$$

where $S_{XY}$ is the joint support. On the other hand, Spearman's Rho can be expressed as

$$\rho_S(X,Y) = 12\cdot \int\int_{S_{XY}}f_x(x)f_y(y)[F_{XY}(x,y) - F_X(x)F_Y(y)] dx dy$$

Those that remember that $dF(x) = f(x)dx$ understand why the existence of densities is not critical. But it is clarifying: compacting $[F_{XY}(x,y) - F_X(x)F_Y(y)] \equiv QD(x,y)$ we have

$$\text{Cov}(X,Y) = \int\int_{S_{XY}}QD(x,y) dx dy$$

$$\rho_S(X,Y) = 12\cdot \int\int_{S_{XY}}f_x(x)f_y(y)QD(x,y) dx dy$$

We see that the covariance "sums" the quantities $QD(x,y)$ over the joint support "unweighted", while Spearman's Rho sums them weighted by the product of the densities, $f_x(x)f_y(y)$ (which is always non-negative). If Quadrant Dependence holds, then in both measures we "sum" either non-negative things only or non-positive things only.

So

a) Under $QD$, Covariance will have the same sign as Spearman's Rho if both are not zero:

$$\text{sign}\left\{\text{Cov}(X,Y)\right\} = \text{sign}\left\{\rho_s(X,Y)\right\}$$

Moreover, consider a strictly increasing monotonic transformation of $Y$, $h(Y)$. Spearmans's Rho is invariant under such a transformation so

$$\rho_S(X,Y) = \rho_S(X,h(Y))$$

Under Quadrant Dependence, we will have, again when both measures are not zero,

$$\text{sign}\left\{\text{Cov}(X,h(Y))\right\} = \text{sign}\left\{\rho_s(X,h(Y))\right\}$$

Linking sign equalities we then obtain

$$\text{sign}\left\{\text{Cov}(X,Y)\right\} = \text{sign}\left\{\text{Cov}(X,h(Y))\right\}$$

As implied in the other answers, the counterintuitive result here is that Quadrant Dependence cannot be dropped: if it does not hold, then we have no guarantee that a strictly increasing transformation of one variable will preserve the sign of covariance. Therefore, "pretty logical" informal arguments like "since, when $Y$ tends to increase so does $h(Y)$, it follows that if $X$ covaries positively with $Y$, it will covary positively also with $h(Y)$" are wrong when by using the verb "co-vary" we have in mind the Covariance measure. "It follows" for Covariance only if $QD$ holds.

Formally, one can see this by setting $Z= h(Y), h'(y) >0$ and observing that

$$F_Z(z) = F_Y(h^{-1}(z)),\;\;\;F_{XZ}(x,z) = F_{XY}(x,h^{-1}(z)), dz = h'(y)dy$$. Then we have

$$\text{Cov}(X,Z) = \int\int_{S_{XZ}}[F_{XZ}(x,z) - F_X(x)F_Z(z)] dx dz$$

$$= \int\int_{S_{XZ}}[F_{XY}(x,h^{-1}(z)) - F_X(x)F_Y(h^{-1}(z))] dx dz$$ and then make a change of variable from $Z$ to $Y$, to get

$$\text{Cov}(X,Z) = \int\int_{S_{X,Y}}h'(y)\cdot QD(x,y)dx dy$$

If $QD$ does not hold, it means that some $QD(x,y)$ will be positive and some negative. Then, the fact that, say $\text{Cov}(X,Y) >0$ alone cannot guarantee that $\text{Cov}(X,Z) >0$ also, since, here, we weight the previous integrand by $h'(y)$, which although strictly positive is not a constant and so it may be the case that it weights disproportionately more those $QD(x,y)$ that are negative, than those that are positive, resulting overall in a negative value. So , from this path at least, the property of Quadrant Dependence is essential.