1
$\begingroup$

The formula for a weighted average is: sum of values, multiplied by respective weights, divided by count of values. Right? That’s what I thought it was until I saw other variations, which are basically the expected value—no dividing by the count of values.

Does the weighted average require dividing the sum of weighted values by their count or is it exactly the same as the expected value formula?

$\endgroup$
  • $\begingroup$ Where did you see the other variations? In what context? See here $\endgroup$ – corey979 Apr 4 at 6:16
  • $\begingroup$ What's the expected value formula you're referring to? There are many ways to write down definitions for expected values. For example, some might use an expectation operator $E()$, defined or assumed familiar, while others might not. $\endgroup$ – Nick Cox Apr 4 at 10:22
  • $\begingroup$ I've also pointed out in the edit of my answer; I've seen that dividing by count is being used in some references (like investopedia), however the common understanding in ML, stats domain is more of alike its wikipedia definition. $\endgroup$ – gunes Apr 4 at 10:23
  • $\begingroup$ @Gunes If weights are presented as adding to some constant not 1, then that is corrected for in any competent calculation. I can't see that there is any debate or dispute here, except a small discussion over stylistic preferences: how to explain what is central and what is detail. I'd rather that definitions on internet sites match best practice in good texts and software, not the other way round, and I think you're saying similarly. The Investopedia page you mention is clearly at the elementary end of the spectrum, and fine by me; I often lurk there myself. $\endgroup$ – Nick Cox Apr 4 at 10:35
3
$\begingroup$

The general idea is that a weighted average (or mean) is for a variable $x$ and weights $w$ $$\bar x = \sum_i w_i x_i / \sum_i w_i.$$

If additionally or alternatively, weights are defined as adding to $1$, say in this notation $w'_i = w_i/\sum_i w_i$, then it follows that you can write $\bar x = \sum_i w'_i x_i$.

The usual unweighted average fits this pattern too. Consider the average $(1 + 2 + 3)/3,$ which we could write in terms of $w_i = 1/3$

$$[(1/3) 1 + (1/3) 2 + (1/3) 3] / [(1/3) + (1/3) + (1/3)]$$

or in terms of $w_i = 1$

$$[1 \times 1 + 1 \times 2 + 1 \times 3] / [1 + 1 + 1]$$

or indeed using any other positive constant $w_i, 42, 666,$ or whatever else takes our fancy.

The more general weighted average is often used without using that name. Suppose the variable is the number of bedrooms per household and in $100 $ households we observe $1$ bedroom $30$ times, $2$ bedrooms $30$ times and $3$ bedrooms $40$ times. Then the appropriate average uses the frequencies as weights, and is thus $(30 + 60 + 120) / 100 = 2.1.$.

The weights do not have to be integers or even counts or frequencies. Thus one simple moving average (in time series analysis, and in some other contexts) is often presented as $0.25 \times$ previous value $+\ 0.5 \times$ present value $+\ 0.25 \times$ next value. That one was called Hanning by John W. Tukey, after Julius von Hann.

There is no requirement that weights are all positive, just that their sum $\sum w_i$ is positive. For example, negative weights arise naturally for certain moving averages in time series analysis. And zero weights are allowed too in a definition, and just result in values not being included at all. An example would be a moving average using a finite window, where implicitly or explicitly observations not in the window get zero weight.

$\endgroup$
  • $\begingroup$ The mathematical definition in wiki page (which is actually a non-scientific source, I know :) ) also states that the weights are non-negative. I understand your reasoning about negative weights, that is a bit reminder of the convolution operation, however such generalization might induce confusions about the weighted arithmetic average definition. $\endgroup$ – gunes Apr 4 at 9:35
  • $\begingroup$ I didn't consult the Wikipedia entry, which looks fine within limits, but what's confusing or misleading here? $\endgroup$ – Nick Cox Apr 4 at 9:55
  • $\begingroup$ Thanks, Nick! This is exactly what I needed--clarity on the general formula and you gave it to me in the first line! Thanks again. $\endgroup$ – Lii Bernovski-Smith Apr 4 at 21:22
  • $\begingroup$ If you accept one answer, you will get a reputation boost. $\endgroup$ – Nick Cox Apr 4 at 22:08
1
$\begingroup$

The trivial case here is the arithmetic average, for example for two numbers we have: $$x_a=\frac{x_1+x_2}{2}=\frac{1}{2}x_1+\frac{1}{2}x_2$$

The denominator in LHS is the number of observations because this is an equal averaging. Everyone equally contributes to the final result. In the RHS, you don't see a denominator, or number of observations, since we embedded the denominator into the weights. The crucial thing is, your final weights need to sum up to $1$, which is why you divide by number of elements in arithmetic means because every summand needs to have weight $1/n$ in the final formulation.

So, it is either $$x_a=\sum w_ix_i, \ \ \ \sum w_i=1\ \ \ \ \text{or} \ \ \ x_a=\frac{\sum w_ix_i}{\sum w_i}$$

In the second one, sum of weights is again $1$ since (call $\sum w_i=W$: $$x_a=\frac{\sum w_ix_i}{W}=\sum\left(\frac{w_i}{W}\right)x_i\rightarrow w_i'=\frac{w_i}{W}$$ And, $$\sum \frac{w_i}{W}=\frac{1}{W}\sum w_i=\frac{W}{W}=1$$

Specifically, the $E[X]$, expected value formulation, includes probabilities: $$E[X]=\sum p_i x_i$$ where $\sum p_i=1$, by the definition of PMFs.

Check also the wiki page provided by @corey979 .

Edit: It seems that interesting definitions exist out in the internet, in which I've never seen being used. It seems like the definition is a very debatable issue, however, I'd stick with the wiki entry, which is closer to the mathematical domain, instead of Investopedia, which uses the term to reach a specific purpose.

$\endgroup$
  • 1
    $\begingroup$ Number of variables would, I suggest, be better as number of observations (or values, or cases, ...). To mind there is only one variable in sight here, namely $x$. $\endgroup$ – Nick Cox Apr 4 at 8:54
  • $\begingroup$ That's correct. Edited right away. Thanks @NickCox $\endgroup$ – gunes Apr 4 at 8:56
  • $\begingroup$ We're not far apart (see my answer), but I don't see that the sum of the weights has to equal 1. You can always rescale weights so that is true, but it's not a requirement. $\endgroup$ – Nick Cox Apr 4 at 9:24
  • $\begingroup$ In the end, with the scalar, the final numbers that are multiplied with the observations will sum up to $1$; otherwise it wouldn't be weighted averaging, right? $\endgroup$ – gunes Apr 4 at 9:32
  • $\begingroup$ You can present that as the algebra, and no objection from me if that is your preference. No program I know well insists that is how the computation must be asked for. Conversely I maintain that my formulation is as natural. I'll edit my answer to wave diplomatically at a formulation like yours. $\endgroup$ – Nick Cox Apr 4 at 9:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.