What is the implication of having the simple mean very different from the expected value?

I was in the process of analyzing the data below. The first 2 columns are correct. They represent a point in time and a value of a stock at that point. I decided to calculate the probability surrounding each value and calculated the average to be the Sum(price)/16 =4.6875. I calculated the mean as Sum(price * P(value=price)=E(y) and took the average of that last column. I expected that these to values to be close but E(y) was found to be 0.7617.

I want to know the following:

1. Do the calculation look right (just take a single random row, I don't expect anyone to waste time checking all the numbers of course).
2. Why is E(y) so different from AVG(y)? What is the statistical significance of that?

The list of y values (2nd column) is:

3, 4, 5, 6, 7, 8, 3, 4, 5, 6, 3, 1, 3, 7, 4, 6


The entire table is:

Time Price.y N.gt.y N.eq.y N.eq.y.1 P.gt.y P.eq.y P.eq.y.1    E.y
1       3     11      4        1 68.75% 25.00%    6.25% 0.7500
2       4      8      3        5 50.00% 18.75%   31.25% 0.7500
3       5      6      2        8 37.50% 12.50%   50.00% 0.6250
4       6      3      3       10 18.75% 18.75%   62.50% 1.1250
5       7      1      2       13  6.25% 12.50%   81.25% 0.8750
6       8      0      1       15  0.00%  6.25%   93.75% 0.5000
7       3     11      4        1 68.75% 25.00%    6.25% 0.7500
8       4      8      3        5 50.00% 18.75%   31.25% 0.7500
9       5      6      2        8 37.50% 12.50%   50.00% 0.6250
10       6      3      3       10 18.75% 18.75%   62.50% 1.1250
11       3     11      4        1 68.75% 25.00%    6.25% 0.7500
12       1     15      1        0 93.75%  6.25%    0.00% 0.0625
13       3     11      4        1 68.75% 25.00%    6.25% 0.7500
14       7      1      2       13  6.25% 12.50%   81.25% 0.8750
15       4      8      3        5 50.00% 18.75%   31.25% 0.7500
16       6      3      3       10 18.75% 18.75%   62.50% 1.1250


• A weighted average won't necessarily equal a simple average. If there is reason to suspect that they should, that could mean that your weights are inaccurate. Can you type / paste in your data to make it easier for people to work with? (Ie, not a jpg; people will help with the formatting.) – gung - Reinstate Monica Jul 28 '16 at 16:17
• Can you explain a bit more about how you arrived at E(y)? – mdewey Jul 28 '16 at 16:35
• @mdewey, E(y) is calculated as follows, say for Time=1, y=3, so E(3)=(P(y=3)*3)=0.25*3=0.75. – NoChance Jul 28 '16 at 17:16
• @gung, thanks for your comment. E(Y) is the weighted average of the cost, I wanted to calculate the Mean as the expected value times the probability. I will post the data in typing now. – NoChance Jul 28 '16 at 17:17

If I understand correctly, it should not be surprising that these numbers are very different. I think you are not comparing what you think you are. From you data I gather you are comparing:

1. The arithmetic mean of the stock price: (sum of prices / n)
2. The arithmetic mean of: (price * the frequency with which that price was observed)

Take row 1 for example. The value 3 was observed 4 times out of 16, or 25% of the time. You took 25% * price (3) which = .75 and labeled .75 as the expected value. Why would .75 be the expected value? According to your frequencies, the expected value will be greater than 3 68.75% of the time and a value less than 3 will be observed 6.25% of the time. Therefore, I would say it is highly unlikely to observe a value of .75.

The mean (.76) at the bottom is the mean if you repeated the above for each row, summed the values, and divided by n (16).

If you want to calculate the average expected value, I recommend you use a simple linear regression with price as your independent variable y and time as your dependent variable x. From there, you can take (sum predicted value y-hat /n) to get the average expected value.

• Thank you for your answer, E(y) in fact is not the correct term. E(y)= Sum (P(yi)*yi) = 3*0.25+4*18.75+6*12.5+...+6*18.75.=0.761. This value is so far from the average 4.6875. Major question is why? What does the large difference signify? – NoChance Jul 28 '16 at 18:53
• There is no good answer to "why" because the numbers you are comparing have no relationship. – James Steele Jul 29 '16 at 16:58

Without repercentaging the P(=y) column to 100%, the average of that column times Price is way underestimating a more appropriate weighted average. This is for the simple reason that the P(=y) column, by itself, does not sum to 100%.

By summing P(=y) up, repercentaging each cell, and multiplying that new value by Price, appropriate weighted average can be generated.

Per @gung's comment, at 4.023, this new weighted average isn't equal to the raw unweighted average.

For me, a more basic question is why you are multiplying price by a partitioned likelihood in the first place.

• Thank you for your answer. I don't think we can sum P(=y) to get 100%. P(v=y) in this context is the probability of the occurrence of the price v in the set given. Since prices repeat, it can't add up to 1. – NoChance Jul 28 '16 at 17:24
• The very premise of your analysis is highly dubious to me. What is your motivation for partitioning the likelihood of a price event into 3 buckets? – Mike Hunter Jul 28 '16 at 17:27
• What I wanted to see is, given a price value such as $6, what is the probability that this price will repeat (=6), increase (>6), decrease (<6). – NoChance Jul 28 '16 at 18:40 • I've been thinking about your reply to my question. I still have questions about the bucketing but there is no information about how you arrived at the likelihoods. My concerns boil down to two: first, why treat the probabilities as mutually exclusive, summing to 100%? They could just as easily be treated as independent, random variables. Next, isn't your primary concern that price could decline? If that is true, then why not just model that? This would eliminate two of the buckets, simplifying the analysis. – Mike Hunter Jul 29 '16 at 18:32 • Thank you for your concern. As per the first question the sum of the values, taken horizontally should be 1 because any number regardless of what it represents must satisfy this$p(x<v)+p(x=v)+p(x>v) = 1$( as long as$x$,$v$are not complex). The vertical sum of probabilities does not have to yield 1. You are correct, the price for the sake of this case, can be considered as a random variable. I was interested in having the full picture for price probabilities, but for the case of price decrease$p(x<=v)\$ would suffice. – NoChance Jul 29 '16 at 19:00