Requesting Intuitive Explanation to Covariance, Correlation and Standard Deviation

Question

First I would like to state that I am not from a mathematical background. I am studying about change in price of products. So I have to understand about Correlation , Covariance and Standard Deviation intuitively. I have read lot of theoretical definitions today and little bit confused.

I have taken the following data as example

When the price of Product X is Rs. 1,2,3,4,5 , the price of product Y is Rs. 3,6,9,12,15.

I use Karl Pearson formula.

The Covariance turns out to be 6 . I dont understand what this number 6 is. My understanding is covariance measures how much Y changes when there is a change in X. But here I thought Covariance should be 3. But it is 6 . So please explain what's wrong with my understanding.

Here the Units of both X and Y is common only. I read that we divide the Covariance by Product SD of X and Y to arrive at a unit free value between X and Y. My first doubt is , what is the intuitive explanation of dividing the Covariance by Product of the SD of two variables.

The correlation turns out to be 1 . My understanding of Correlation is it measures the strength and not the slope. When correlation is 1 it means , when X changes in one direction , Y always changes in the same direction . When Corrrelation is 0.9 it means 90 % of the time the variables move in the same direction.

And final doubt is regarding Standard Deviation. I understand SD as average distance of each variable from mean. So why don't we take the absolute value of differences from mean and average it. I heard this wont be accurate if the data's spread is high and I understand it intitively. But I dont get the derivation of formula mathematically.

I would be really really grateful if someone could explain the concept with numbers instead of theory ( would love if it is based on my example data) in a simple way to a non-math student like me. !!

Answer 1

First, I will use some notation to make things easier:

• $SD(X) = $ standard deviation of $X$
• $SD(Y) = $ standard deviation of $Y$
• $Var(X) = $ variance of $X = SD(X) \times SD(X)$
• $Cov(X,Y) = $ covariance between $X$ and $Y$
• $Cor(X,Y) = $ correlation between $X$ and $Y$

Now that you have that down, I should explain what these things mean.

• Standard Deviation is the measure of how far values deviate from their average value. A low standard deviation indicates that data points tend to be close to the average. A high standard deviation indicates that data points tend to be far away from the average.
• Covariance is the measure of "joint variability" between two variables (X and Y in this case). Positive covariance means that when values of X increase, values of Y generally also increase. Negative covariance means that when values of X increase, values of Y generally decrease. Zero covariance means that when the values of X increase, this has no effect on Y.
• Correlation is a standardized way of thinking about covariance.

Now on to formulas, then on to your example. $$ N = \text{number in sample from your population} \\ \mu_X = \text{average of X} \\ Var(X) = \frac{1}{N} \times \sum_{i=1}^N (X_i - \mu_X)^2 \\ $$

In your sample, for $N_X = 5$. This means there are 5 observations of $X$ in your sample. $\mu_X$ is the average of $X$, which is calculated like this:

$$ \mu_X = \frac{(1 + 2 + 3 + 4 + 5)}{N_X} = \frac{(1 + 2 + 3 + 4 + 5)}{5} = \frac{15}{5} = 3 = \mu_X $$

Now, to calculate variance of X, we use the formula I stated above, which uses summation notation. I will show you the calculation here:

$$ Var(X) = \frac{1}{5} \times \{ (1 - 3)^2 + (2 - 3)^2 + (3 - 3)^2 + (4 - 3)^2 + (5 - 3)^2\} \\ = \frac{1}{5} \times \{ 4 + 1 + 0 + 1 + 4 \} = \frac{1}{5} \times 10 = 2 = Var(X) $$

Now that we have the variance of $X$, we can easily get the standard deviation of $X$, since we know that $[SD(X)]^2 = Var(X)$.

$$ SD(X) = \sqrt{Var(X)} = \sqrt{2} = SD(X) $$

Now you have the standard deviation of X. We need to get $SD(Y)$ in the same way:

$$ \mu_Y = \frac{(3+6+9+12+15)}{N_Y} = \frac{(3+6+9+12+15)}{5} = \frac{45}{5} = 9 = \mu_Y $$

$$ Var(Y) = \frac{1}{5} \times \{ (3 - 9)^2 + (6 - 9)^2 + (9 - 9)^2 + (12 - 9)^2 + (15 - 9)^2\} \\ = \frac{1}{5} \times \{ 36 + 9 + 0 + 9 + 36 \} = \frac{1}{5} \times 90 = \frac{90}{5} = 18 = Var(Y) $$

$$ SD(Y) = \sqrt{Var(Y)} = \sqrt{Var(Y)} = \sqrt{18} $$

Now we move on to covariance, which uses the following formula:

$$ Cov(X,Y) = \frac{1}{N} \times \sum_{i=1}^N (X_i - \mu_X)\times(Y_i - \mu_i) $$

This seems tricky, but here is the arithmetic:

$$ Cov(X,Y) = \frac{1}{5} \times \{ (1 - 3)(3 - 9) + (2 - 3)(6 - 9) + (3 - 3)(9 - 9) + (4 - 3)(12 - 9) + (5 - 3)(15 - 9) \} \\ = \frac{1}{5} \times \{ (-2)\times(-6) + (-1)\times(-3) + (0)\times(0) + (1)\times(3) + (2)\times(6) \} \\ = \frac{1}{5} \times \{ 12 + 3 + 0 + 3 + 12 \} = \frac{1}{5} \times 30 = 6 = Cov(X,Y) $$

We are almost done! Now we need to calculate correlation, which is just a standardized way to represent covariance that lets us compare two variables with different sized numbers (correlation of ear size to height; sure, they will be correlated, but height will vary a lot more than ear size because the units of height are much larger, causing a larger variance).

$$ Cor(X,Y) = \frac{Cov(X,Y)}{SD(X)\times SD(Y)} $$

We know all of the above values, so we can calculate straight-away!

$$ Cor(X,Y) = \frac{6}{\sqrt{2} \times \sqrt{18}} = \frac{6}{\sqrt{2 \times 18}} = \frac{6}{\sqrt{36}} = \frac{6}{6} = 1 = Cor(X,Y) $$

This result makes sense, as we see that every time $X$ increases by 1, $Y$ increases by 3. This happens for every movement, meaning that every time $X$ moves, $Y$ will respond linearly, in the same amount, in a perfectly correlated fashion.

There you have it, Variance 101.