Requesting Intuitive Explanation to Covariance, Correlation and Standard Deviation

First I would like to state that I am not from a mathematical background. I am studying about change in price of products. So I have to understand about Correlation , Covariance and Standard Deviation intuitively. I have read lot of theoretical definitions today and little bit confused.

I have taken the following data as example

When the price of Product X is Rs. 1,2,3,4,5 , the price of product Y is Rs. 3,6,9,12,15.

I use Karl Pearson formula.

The Covariance turns out to be 6 . I dont understand what this number 6 is. My understanding is covariance measures how much Y changes when there is a change in X. But here I thought Covariance should be 3. But it is 6 . So please explain what's wrong with my understanding.

Here the Units of both X and Y is common only. I read that we divide the Covariance by Product SD of X and Y to arrive at a unit free value between X and Y. My first doubt is , what is the intuitive explanation of dividing the Covariance by Product of the SD of two variables.

The correlation turns out to be 1 . My understanding of Correlation is it measures the strength and not the slope. When correlation is 1 it means , when X changes in one direction , Y always changes in the same direction . When Corrrelation is 0.9 it means 90 % of the time the variables move in the same direction.

And final doubt is regarding Standard Deviation. I understand SD as average distance of each variable from mean. So why don't we take the absolute value of differences from mean and average it. I heard this wont be accurate if the data's spread is high and I understand it intitively. But I dont get the derivation of formula mathematically.

I would be really really grateful if someone could explain the concept with numbers instead of theory ( would love if it is based on my example data) in a simple way to a non-math student like me. !!

• "But here I thought Covariance" should be 3. You need to explain why you think this. But it is easy to guess where you got 3. In your simple example $Y = 3X$ exactly. But 3 isn't the covariance at all. It's called the coefficient of $X$ or the slope or the gradient of the line predicting $Y$ from $X$. – Nick Cox Jul 9 '18 at 13:08
• "When Corrrelation is 0.9 it means 90 % of the time the variables move in the same direction." No; that is not the definition. Occasionally it's correct: for example, if the correlation is 1 then 100% of changes are in the same direction. – Nick Cox Jul 9 '18 at 13:12
• This is, or should be, well covered in every decent introductory text (so, downvote from me for lack of research effort). That doesn't rule out a thread here, but it does rule in the possibility of giving you some references. Freedman, Pisani, Purves Statistics New York: WW Norton (any of various editions from 1978 to 2007) is one excellent source. – Nick Cox Jul 9 '18 at 13:42
• Thanks a lot Mr.Nick Cox. Its good that you pointed out my wrong comparison of Slope and Covariance. And relating to the 90 % thing , then what does a correlation of 0.9 means intuitively sir ? If +1 correlation means , 100 % of changes in same direction..Then +0.9 means ? Will surely go through the referred book . Will be grateful if you could answer my above question alone and what does covariance of 6 represent here intuitively with my example numbers if possible. Sorry if I demand a lot from you. Will try to giveback manyfold once I understand the concepts. – Emperor Jul 9 '18 at 15:46
• Sorry, but as commented elsewhere I have no interest in reproducing basic textbook material in a full answer. But two points: A correlation such as 0.9 isn't to be interpreted as a fraction of anything, if only because (not only because) correlations can be negative as well as positive. Correlation is correlation, not something else in disguise, except perhaps a cosine measuring the similarity between vectors (but my guess is that is more mathematical than you want, but it's highly intuitive to anyone with a certain mathematical background). – Nick Cox Jul 9 '18 at 15:55

First, I will use some notation to make things easier:

• $SD(X) =$ standard deviation of $X$
• $SD(Y) =$ standard deviation of $Y$
• $Var(X) =$ variance of $X = SD(X) \times SD(X)$
• $Cov(X,Y) =$ covariance between $X$ and $Y$
• $Cor(X,Y) =$ correlation between $X$ and $Y$

Now that you have that down, I should explain what these things mean.

• Standard Deviation is the measure of how far values deviate from their average value. A low standard deviation indicates that data points tend to be close to the average. A high standard deviation indicates that data points tend to be far away from the average.
• Covariance is the measure of "joint variability" between two variables (X and Y in this case). Positive covariance means that when values of X increase, values of Y generally also increase. Negative covariance means that when values of X increase, values of Y generally decrease. Zero covariance means that when the values of X increase, this has no effect on Y.
• Correlation is a standardized way of thinking about covariance.

Now on to formulas, then on to your example. $$N = \text{number in sample from your population} \\ \mu_X = \text{average of X} \\ Var(X) = \frac{1}{N} \times \sum_{i=1}^N (X_i - \mu_X)^2 \\$$

In your sample, for $N_X = 5$. This means there are 5 observations of $X$ in your sample. $\mu_X$ is the average of $X$, which is calculated like this:

$$\mu_X = \frac{(1 + 2 + 3 + 4 + 5)}{N_X} = \frac{(1 + 2 + 3 + 4 + 5)}{5} = \frac{15}{5} = 3 = \mu_X$$

Now, to calculate variance of X, we use the formula I stated above, which uses summation notation. I will show you the calculation here:

$$Var(X) = \frac{1}{5} \times \{ (1 - 3)^2 + (2 - 3)^2 + (3 - 3)^2 + (4 - 3)^2 + (5 - 3)^2\} \\ = \frac{1}{5} \times \{ 4 + 1 + 0 + 1 + 4 \} = \frac{1}{5} \times 10 = 2 = Var(X)$$

Now that we have the variance of $X$, we can easily get the standard deviation of $X$, since we know that $[SD(X)]^2 = Var(X)$.

$$SD(X) = \sqrt{Var(X)} = \sqrt{2} = SD(X)$$

Now you have the standard deviation of X. We need to get $SD(Y)$ in the same way:

$$\mu_Y = \frac{(3+6+9+12+15)}{N_Y} = \frac{(3+6+9+12+15)}{5} = \frac{45}{5} = 9 = \mu_Y$$

$$Var(Y) = \frac{1}{5} \times \{ (3 - 9)^2 + (6 - 9)^2 + (9 - 9)^2 + (12 - 9)^2 + (15 - 9)^2\} \\ = \frac{1}{5} \times \{ 36 + 9 + 0 + 9 + 36 \} = \frac{1}{5} \times 90 = \frac{90}{5} = 18 = Var(Y)$$

$$SD(Y) = \sqrt{Var(Y)} = \sqrt{Var(Y)} = \sqrt{18}$$

Now we move on to covariance, which uses the following formula:

$$Cov(X,Y) = \frac{1}{N} \times \sum_{i=1}^N (X_i - \mu_X)\times(Y_i - \mu_i)$$

This seems tricky, but here is the arithmetic:

$$Cov(X,Y) = \frac{1}{5} \times \{ (1 - 3)(3 - 9) + (2 - 3)(6 - 9) + (3 - 3)(9 - 9) + (4 - 3)(12 - 9) + (5 - 3)(15 - 9) \} \\ = \frac{1}{5} \times \{ (-2)\times(-6) + (-1)\times(-3) + (0)\times(0) + (1)\times(3) + (2)\times(6) \} \\ = \frac{1}{5} \times \{ 12 + 3 + 0 + 3 + 12 \} = \frac{1}{5} \times 30 = 6 = Cov(X,Y)$$

We are almost done! Now we need to calculate correlation, which is just a standardized way to represent covariance that lets us compare two variables with different sized numbers (correlation of ear size to height; sure, they will be correlated, but height will vary a lot more than ear size because the units of height are much larger, causing a larger variance).

$$Cor(X,Y) = \frac{Cov(X,Y)}{SD(X)\times SD(Y)}$$

We know all of the above values, so we can calculate straight-away!

$$Cor(X,Y) = \frac{6}{\sqrt{2} \times \sqrt{18}} = \frac{6}{\sqrt{2 \times 18}} = \frac{6}{\sqrt{36}} = \frac{6}{6} = 1 = Cor(X,Y)$$

This result makes sense, as we see that every time $X$ increases by 1, $Y$ increases by 3. This happens for every movement, meaning that every time $X$ moves, $Y$ will respond linearly, in the same amount, in a perfectly correlated fashion.

There you have it, Variance 101.

• Helpful, but I don't think good intuition can be imparted without explaining more about the units of measurement. Even here, the covariance of a relationship of prices has unusual units (squared Rupees in the OP's example). – Nick Cox Jul 9 '18 at 13:14
• "Zero covariance means that when the values of X increase, this has no effect on Y." As you'll realise that's an oversimplification as zero covariance can arise in different ways. – Nick Cox Jul 9 '18 at 13:15
• Yes, it is an oversimplification, by design! OP wanted an explanation of the calculations, which I gave, and a non-techhnical overview of variance, which I attempted to give. – ERT Jul 9 '18 at 13:18
• We agree and (candidly) I have no interest in writing out an answer myself when it would be what every decent text explains. I upvoted your answer but if you're tempted to amplify it then I think you need to address the OP's "The Covariance turns out to be 6 . I don't understand what this number 6 is." – Nick Cox Jul 9 '18 at 13:39
• Thank you Trauger for your nice explanation . Can I please know what does Covariance of 6 actually represent here and why we divide by SD(X) and SD(Y) to standardize it . Kindly help me with an intuitive explanation – Emperor Jul 9 '18 at 15:50