17
votes
If Cov(X,Y)=Var(Y), what is the dependence between X and Y?
If we let $\rho_{X,Y}$ denote the correlation between the variables and let $s_X$ and $s_Y$ denote their respective standard deviations then we have:
$$\rho_{X,Y} s_X s_Y = \mathbb{Cov}(X,Y) = \mathbb{...
- 130k
16
votes
If Cov(X,Y)=Var(Y), what is the dependence between X and Y?
Concerning the question of linear dependence, you can conclude only that the variables are positively related.
From the usual formulas, with $s_*$ representing standard deviations and $\rho$ the (...
- 330k
15
votes
Accepted
How was the correlation coefficient formula derived?
Too keep things as simple as possible, assume that $\mathbf{x} = (1,2)$ and $\mathbf{y} = (3,-5)$. You will think of $\mathbf{x},\mathbf{y}$ as "vectors", i.e. as arrows that start at the ...
- 3,193
14
votes
Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?
If $X \mathrel{:=} \left(X_1, \ldots, X_k\right)^\top \sim \mathop{\mathrm{Multinomial}}\left(n, \left(p_1, \ldots, p_k\right)^\top\right)$ with $n \in \mathbb N_{\geq 1}, k \in \mathbb N_{\geq 2},$ ...
- 6,355
13
votes
Accepted
Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?
Mathematically, as I commented, you can use the equi-correlation matrix
\begin{align*}
\Sigma =
\begin{bmatrix}
1 & \rho & \cdots & \rho \\
\rho & 1 & \cdots & \rho \\
\vdots ...
- 20.6k
13
votes
Accepted
Dropping outlier from linear regression model reducing adjusted R^2
Seems to be a situation like in the well known Anscombe quartet, the lower right graph with Y4. Probably your situation is less dramatic, in the sense the vertical bar of 9 points (at the left in that ...
- 1,818
12
votes
If Cov(X,Y)=Var(Y), what is the dependence between X and Y?
If $Var(Y) = 0$, then the statement $Cov(X,Y)=Var(Y)$ holds for all variables $X$, so let's consider only the case where $Var(Y)>0$.
Considering $X$ and $Y$ as vectors, we have that $X \cdot Y = Y \...
- 5,246
12
votes
Which correlation analysis method should be prioritized when different methods yield completely different rankings?
I have to say that this question is, to me, posed backwards, for several quite different reasons. Here are some.
Only exceptionally can different correlations be expected to agree, as they are based ...
- 58.7k
11
votes
Accepted
Should I use Pearson, Kendall, or Spearman correlation?
You wrote
my response variable, 'disease_severity,' is non-linear.
A variable cannot be linear or non-linear. That is a characteristic of relationships between variables.
The Spearman method seems ...
- 125k
11
votes
Accepted
Pearson correlation as a metric for the quality of regression models
I take it that correlation here is the correlation between observed and predicted outcome or response values. If so, then using such a correlation is just a variant on using the coefficient of ...
- 58.7k
11
votes
Count predictor and binary outcome
Per your questions...
Is a binary logistic regression the best approach when I have a count predictor and a binary outcome?
Yes. Logistic regression handles any linear equation which requires the ...
- 16.2k
11
votes
Accepted
What does it mean for observations to be uncorrelated and have constant variance?
These are assumptions made for certain models to ensure certain properties, like valid test statistics. There's a great overview here. The key word here is assumption. These need not hold up in real ...
- 14.2k
10
votes
Which correlation analysis method should be prioritized when different methods yield completely different rankings?
I agree with the answers by Frank and Nick. Adding to them:
Could you advise on the most suitable correlation method for a general
analysis?
No, and I'd be leery of anyone who did so. Frank's answer ...
- 125k
10
votes
Accepted
What is it called when two variables causally affect one another?
I've seen the terms mutual causation, bidirectional causation, and reciprocal causation used. I'm not sure whether there is a standard term in statistics or whether these are different terms meaning ...
- 19.8k
10
votes
What does it mean for observations to be uncorrelated and have constant variance?
Random variables VS observations.
Strictly speaking, there are random variables (which take values in $\mathbb{R}$) and realizations of these random variables (which are elements of $\mathbb{R}$). ...
- 421
9
votes
What does reported "r" mean in the context of a t-test?
Without any other information, I would say $r$ is correlation. And that is an effect size.
- 125k
9
votes
Accepted
What does reported "r" mean in the context of a t-test?
The reported $r$ is usually referring to an effect size. It is calculated from the t-statistic and the degrees of freedom
$$
r = \sqrt{\frac{t^2}{t^2 + df}}
$$
where $t$ is the t-statistic and $df$ is ...
- 63.6k
9
votes
Which correlation analysis method should be prioritized when different methods yield completely different rankings?
When the variables are almost continuous and relationships are not expected to be very non-monotonic my default choice is Spearman’s $\rho$. If you expect non-monotonic relationships that do not ...
- 96.4k
9
votes
Pearson correlation as a metric for the quality of regression models
As Nick Cox pointed out, Pearson correlation between true and predicted values has an equivalence with the $R^2$ of classical linear regression.
The trouble I see is that multiple expressions are ...
- 65.7k
9
votes
Correlation for Small Dataset?
Estimators based on sample statistics are not very reliable indicators of the population quantities they estimate when sample sizes are very small.
In short, the sample correlation $r$ could be very ...
- 287k
8
votes
Accepted
Is there a simpler proof than mine for this obvious proposition about correlations?
Let $X_i$ be independent random variables with the same distribution as $X.$ Because $g$ is weakly increasing if and only if $(g(x_2)-g(x_1))(x_2-x_1)\ge 0$ for all real numbers $x_i,$
$$\...
- 330k
8
votes
Accepted
What affects correlation in this situation?
I would have said a sample of $20$ observations rather than $20$ samples.
Suppose the four states are the four visible clusters in this scatterplot:
In each state separately there is zero correlation,...
- 10.7k
8
votes
Accepted
Imaginary numbers in PCA output
A correlation matrix is symmetrical and hence all the eigenvalues are real.
I would verify a few things:
I would check if the correlation matrix is indeed a symmetric matrix.
Is the magnitude large ...
- 6,966
8
votes
What is the formula for the conditional inverse function for the Ali-Mikhail-Haq and the Farlie-Gumbel-Morgenstern Copulas?
By definition, when $(U,V)$ has a copula distribution $C,$
$$C(u,v) = \Pr(U\le u,\ V \le v).$$
To find the distribution conditional on $u$ when $C$ is differentiable at $u$ with derivative $C_u(u,v)$ ...
- 330k
8
votes
Test for multicollinearity with binary and continuous independent variables
Correlations are not good direct approximations of collinearity. The issue is that they only consider the covariance between two variables and not the entire model. The usual go-to is the variance ...
- 16.2k
8
votes
Test for multicollinearity with binary and continuous independent variables
Shawn is right that correlations are not a good approximation to collinearity (although they are often used for that); he also gives the reason for that. He is also right that VIF is the usual method. ...
- 125k
8
votes
Count predictor and binary outcome
Is a binary logistic regression the best approach when I have a count
predictor and a binary outcome?
It is certainly one valid approach, probably the most common one. Is it "best"? That ...
- 125k
7
votes
Accepted
Minimum Pearson's correlation between $X$ and sign($X$)$\cdot X^2$
For any random variable (regardless continuous or discrete) that is symmetric about $0$, since $\operatorname{Cov}(X, \operatorname{sign}(X)X^2) = E[|X|^3] \geq 0$, it can be seen that the correlation ...
- 20.6k
7
votes
Is this a correlation request?
First, I would expand it to 13 columns.
Then, while you could do some measure of association for each combination of products, that would give you $12\times 11/2 = 66$ measures. Hard to interpret, ...
- 125k
7
votes
If Cov(X,Y)=Var(Y), what is the dependence between X and Y?
In his masterly answer (now revised), whuber shows that with regard to the question of linear dependence between $X$ and $Y$, the only conclusion that can be drawn from the hypothesis that $\...
- 47.3k
Only top scored, non community-wiki answers of a minimum length are eligible