First I would like to state that I am not from a mathematical background. I am studying about change in price of products. So I have to understand about Correlation , Covariance and Standard Deviation intuitively. I have read lot of theoretical definitions today and little bit confused.

I have taken the following data as example

When the price of Product X is Rs. 1,2,3,4,5 , the price of product Y is Rs. 3,6,9,12,15.

I use Karl Pearson formula.

The Covariance turns out to be 6 . I dont understand what this number 6 is. My understanding is covariance measures how much Y changes when there is a change in X. But here I thought Covariance should be 3. But it is 6 . So please explain what's wrong with my understanding.

Here the Units of both X and Y is common only. I read that we divide the Covariance by Product SD of X and Y to arrive at a unit free value between X and Y. My first doubt is , what is the intuitive explanation of dividing the Covariance by Product of the SD of two variables.

The correlation turns out to be 1 . My understanding of Correlation is it measures the strength and not the slope. When correlation is 1 it means , when X changes in one direction , Y always changes in the same direction . When Corrrelation is 0.9 it means 90 % of the time the variables move in the same direction.

And final doubt is regarding Standard Deviation. I understand SD as average distance of each variable from mean. So why don't we take the absolute value of differences from mean and average it. I heard this wont be accurate if the data's spread is high and I understand it intitively. But I dont get the derivation of formula mathematically.

I would be really really grateful if someone could explain the concept with numbers instead of theory ( would love if it is based on my example data) in a simple way to a non-math student like me. !!

  • "But here I thought Covariance" should be 3. You need to explain why you think this. But it is easy to guess where you got 3. In your simple example $Y = 3X$ exactly. But 3 isn't the covariance at all. It's called the coefficient of $X$ or the slope or the gradient of the line predicting $Y$ from $X$. – Nick Cox Jul 9 at 13:08
  • "When Corrrelation is 0.9 it means 90 % of the time the variables move in the same direction." No; that is not the definition. Occasionally it's correct: for example, if the correlation is 1 then 100% of changes are in the same direction. – Nick Cox Jul 9 at 13:12
  • This is, or should be, well covered in every decent introductory text (so, downvote from me for lack of research effort). That doesn't rule out a thread here, but it does rule in the possibility of giving you some references. Freedman, Pisani, Purves Statistics New York: WW Norton (any of various editions from 1978 to 2007) is one excellent source. – Nick Cox Jul 9 at 13:42
  • Thanks a lot Mr.Nick Cox. Its good that you pointed out my wrong comparison of Slope and Covariance. And relating to the 90 % thing , then what does a correlation of 0.9 means intuitively sir ? If +1 correlation means , 100 % of changes in same direction..Then +0.9 means ? Will surely go through the referred book . Will be grateful if you could answer my above question alone and what does covariance of 6 represent here intuitively with my example numbers if possible. Sorry if I demand a lot from you. Will try to giveback manyfold once I understand the concepts. – Emperor Jul 9 at 15:46
  • Sorry, but as commented elsewhere I have no interest in reproducing basic textbook material in a full answer. But two points: A correlation such as 0.9 isn't to be interpreted as a fraction of anything, if only because (not only because) correlations can be negative as well as positive. Correlation is correlation, not something else in disguise, except perhaps a cosine measuring the similarity between vectors (but my guess is that is more mathematical than you want, but it's highly intuitive to anyone with a certain mathematical background). – Nick Cox Jul 9 at 15:55

First, I will use some notation to make things easier:

  • $SD(X) = $ standard deviation of $X$
  • $SD(Y) = $ standard deviation of $Y$
  • $Var(X) = $ variance of $X = SD(X) \times SD(X)$
  • $Cov(X,Y) = $ covariance between $X$ and $Y$
  • $Cor(X,Y) = $ correlation between $X$ and $Y$

Now that you have that down, I should explain what these things mean.

  • Standard Deviation is the measure of how far values deviate from their average value. A low standard deviation indicates that data points tend to be close to the average. A high standard deviation indicates that data points tend to be far away from the average.
  • Covariance is the measure of "joint variability" between two variables (X and Y in this case). Positive covariance means that when values of X increase, values of Y generally also increase. Negative covariance means that when values of X increase, values of Y generally decrease. Zero covariance means that when the values of X increase, this has no effect on Y.
  • Correlation is a standardized way of thinking about covariance.

Now on to formulas, then on to your example. $$ N = \text{number in sample from your population} \\ \mu_X = \text{average of X} \\ Var(X) = \frac{1}{N} \times \sum_{i=1}^N (X_i - \mu_X)^2 \\ $$

In your sample, for $N_X = 5$. This means there are 5 observations of $X$ in your sample. $\mu_X$ is the average of $X$, which is calculated like this:

$$ \mu_X = \frac{(1 + 2 + 3 + 4 + 5)}{N_X} = \frac{(1 + 2 + 3 + 4 + 5)}{5} = \frac{15}{5} = 3 = \mu_X $$

Now, to calculate variance of X, we use the formula I stated above, which uses summation notation. I will show you the calculation here:

$$ Var(X) = \frac{1}{5} \times \{ (1 - 3)^2 + (2 - 3)^2 + (3 - 3)^2 + (4 - 3)^2 + (5 - 3)^2\} \\ = \frac{1}{5} \times \{ 4 + 1 + 0 + 1 + 4 \} = \frac{1}{5} \times 10 = 2 = Var(X) $$

Now that we have the variance of $X$, we can easily get the standard deviation of $X$, since we know that $[SD(X)]^2 = Var(X)$.

$$ SD(X) = \sqrt{Var(X)} = \sqrt{2} = SD(X) $$

Now you have the standard deviation of X. We need to get $SD(Y)$ in the same way:

$$ \mu_Y = \frac{(3+6+9+12+15)}{N_Y} = \frac{(3+6+9+12+15)}{5} = \frac{45}{5} = 9 = \mu_Y $$

$$ Var(Y) = \frac{1}{5} \times \{ (3 - 9)^2 + (6 - 9)^2 + (9 - 9)^2 + (12 - 9)^2 + (15 - 9)^2\} \\ = \frac{1}{5} \times \{ 36 + 9 + 0 + 9 + 36 \} = \frac{1}{5} \times 90 = \frac{90}{5} = 18 = Var(Y) $$

$$ SD(Y) = \sqrt{Var(Y)} = \sqrt{Var(Y)} = \sqrt{18} $$

Now we move on to covariance, which uses the following formula:

$$ Cov(X,Y) = \frac{1}{N} \times \sum_{i=1}^N (X_i - \mu_X)\times(Y_i - \mu_i) $$

This seems tricky, but here is the arithmetic:

$$ Cov(X,Y) = \frac{1}{5} \times \{ (1 - 3)(3 - 9) + (2 - 3)(6 - 9) + (3 - 3)(9 - 9) + (4 - 3)(12 - 9) + (5 - 3)(15 - 9) \} \\ = \frac{1}{5} \times \{ (-2)\times(-6) + (-1)\times(-3) + (0)\times(0) + (1)\times(3) + (2)\times(6) \} \\ = \frac{1}{5} \times \{ 12 + 3 + 0 + 3 + 12 \} = \frac{1}{5} \times 30 = 6 = Cov(X,Y) $$

We are almost done! Now we need to calculate correlation, which is just a standardized way to represent covariance that lets us compare two variables with different sized numbers (correlation of ear size to height; sure, they will be correlated, but height will vary a lot more than ear size because the units of height are much larger, causing a larger variance).

$$ Cor(X,Y) = \frac{Cov(X,Y)}{SD(X)\times SD(Y)} $$

We know all of the above values, so we can calculate straight-away!

$$ Cor(X,Y) = \frac{6}{\sqrt{2} \times \sqrt{18}} = \frac{6}{\sqrt{2 \times 18}} = \frac{6}{\sqrt{36}} = \frac{6}{6} = 1 = Cor(X,Y) $$

This result makes sense, as we see that every time $X$ increases by 1, $Y$ increases by 3. This happens for every movement, meaning that every time $X$ moves, $Y$ will respond linearly, in the same amount, in a perfectly correlated fashion.

There you have it, Variance 101.

  • Helpful, but I don't think good intuition can be imparted without explaining more about the units of measurement. Even here, the covariance of a relationship of prices has unusual units (squared Rupees in the OP's example). – Nick Cox Jul 9 at 13:14
  • "Zero covariance means that when the values of X increase, this has no effect on Y." As you'll realise that's an oversimplification as zero covariance can arise in different ways. – Nick Cox Jul 9 at 13:15
  • Yes, it is an oversimplification, by design! OP wanted an explanation of the calculations, which I gave, and a non-techhnical overview of variance, which I attempted to give. – ERT Jul 9 at 13:18
  • We agree and (candidly) I have no interest in writing out an answer myself when it would be what every decent text explains. I upvoted your answer but if you're tempted to amplify it then I think you need to address the OP's "The Covariance turns out to be 6 . I don't understand what this number 6 is." – Nick Cox Jul 9 at 13:39
  • Thank you Trauger for your nice explanation . Can I please know what does Covariance of 6 actually represent here and why we divide by SD(X) and SD(Y) to standardize it . Kindly help me with an intuitive explanation – Emperor Jul 9 at 15:50

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.