3
$\begingroup$

I'm reading Linear Regression and correlation by Scott Hartshorn. There is Regression example for a television show Modern Family. We calculated equation for a regression line to predict number of viewers of that show. Then there is the example predicting errors between predicted number of viewers and actual value. So we have

____________________________________________________________________________
|Normalized   |      Predicted normalized               |   Regression error |
|viewers (Nv) |  viewers with our Regression line(PNv)  |      (PNv - Nv)    |
|____________________________________________________________________________|
|1            |              0.915                      |        -0.085      |
|0.79         |              0.907                      |         0.114      |
|...          |              ...                        |         ...        |
|_____________|_________________________________________|____________________|

We made a histogram of the regression errors.

enter image description here enter image description here enter image description here

Then the author drew normal curve(it is on the histogram). And I don't understand the formula is being used to draw that curve.

We have standard deviation of Regression errors = 0.074 There is table with data for provided histogram above.

_______________________________________________________________________________
Histogram bin index   |    Histogram bins |          Normal curve              |
______________________|___________________|____________________________________|
       1              |         -0.2      |      =NORM.S.DIST(-0.2/0.074, 0)*  |
                      |                   |   Number_of_episodes*(0.02/0.074)=0.458  |
______________________|___________________|____________________________________|
       2              |         -0.18     |      =NORM.S.DIST(-0.18/0.074, 0)* |
                      |                   |   Number_of_episodes*(0.02/0.074)=0.9194  |
       ...            |         ...       |            ...                     |
______________________|___________________|____________________________________|

Per my understanding normal curve consists of product of 3 parts:

  1. NORM.S.DIST(-0.2/0.074, 0) That excel function is used to calculate probability density function value for standard distribution. -0.2/0.074 - Z score and 0 - is just a marker that we need to use density.

2.Number_of_episodes as far as I understand it is size of sample. It is sum of values from y axis on the histogram above.

3.(0.02/0.074) not sure what it is. Based on the table and graph above, 0.02 is distance between bins on histogram. 0.074 is standard deviation.

Can you please explain what is the formula being used by author to calculate normal curve? I though that NORM.S.DIST(-0.2/0.074, 0) is enough.

I'm kind of newbie in statistic and probability, sorry if I missed some data to be shown. I will edit the message and add it.

$\endgroup$

1 Answer 1

1
$\begingroup$

A normal distribution is completely characterized by its mean $\mu$ and its standard deviation $\sigma$ (equivalently, its variance $\sigma^2.$) Once you compute the mean (I assume you know how to compute that) and the (sample) standard deviation: $$\sigma=\sqrt{\frac{1}{N-1}\sum_{i=1}^N(y_i-\overline{y})^2},$$ then the formula for a normal distribution is $$f(y)=\frac{1}{\sigma\sqrt{2\pi}}\,\exp\left[-\frac{(y-\mu)^2}{2\sigma^2}\right].$$ However, as whuber pointed out in his comment, the histogram shown is not that of a distribution, since the area underneath it is clearly greater than $1.$ To get an approximate height for the curve, you can simply multiply the formula above by the sum of all the values (an approximation to the area under the curve) times the horizontal differential: $$\tilde{f}(y)=\frac{1}{\sigma\sqrt{2\pi}}\,\exp\left[-\frac{(y-\mu)^2}{2\sigma^2}\right]\times\Delta x\times\sum_{i=1}^ny_i.$$

This is my guess for how the author made the normal curve.

$\endgroup$
10
  • $\begingroup$ Thanks for the answer. I tried to play with that formula. But It doesn't work for me. In fact the probability density function is used in Excel function NORM.S.DIST with mean equal 0 and standard deviation equal to 1. $\endgroup$
    – Dron4K
    Jul 16, 2021 at 21:56
  • $\begingroup$ I'm not sure what you mean by "But it doesn't work for me." Can you clarify? $\endgroup$ Jul 16, 2021 at 21:58
  • 1
    $\begingroup$ The curve is adjusted to reflect (a) a particular standard deviation and (b) normalization to a total count rather than a unit probability. A full explanation should include both these points. $\endgroup$
    – whuber
    Jul 16, 2021 at 22:00
  • $\begingroup$ @Adrian I tried to calculate "Normal curve" values in the table with f(x)=(1/σ√2π)*exp(-((x-μ)^2)/2σ^2) So I used standard deviation - 0.074(standard deviation of errors); x - is value from x axis on histogram(e.g. 0.2); μ - zero. I got 0.137955 what is different of author's value(I updated my message) $\endgroup$
    – Dron4K
    Jul 16, 2021 at 22:08
  • 2
    $\begingroup$ Agree with $\mu=0$ and $\sigma\approx 0.074.$ See the edit for "de-normalizing", so-to-speak, as whuber pointed out. $\endgroup$ Jul 16, 2021 at 22:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.