1
$\begingroup$

I have ~16,000 probability sets of goals scored [maximum of 12] like below [some % are rounded hence do not add up to 100%]:

Goals  0    1    2    3    4    5    6    7    8    9   10   11   12
-------------------------------------------------------------------------
1:     9   15   24   23   14    8    4    2  0.5  0.2...    AVERAGE TEAM
2:    29   25   26   11    -    -    -    9    -    -    -
3:     4    9   32   33   10   10    2    1    -    -    -
4:    22   36   17   16    9    -    -    -    -    -    -  BAD TEAM
5:     3   10   11   22   23   19    9    1    1    -    -  GOOD TEAM

Tacky illustrative examples of objective [on the gentle side]:
2:   27   29   24   10    1    -    1    7    1    -    -  
3:    4   11   30   31   13    8    3    1    -    -    -
4:   No problem other than 5 & 6 goals should perhaps greater than 0%
  • 2: Is a perfect example: 9% for 7 goals but 0% for 4, 5, 6 & 8 goals is a serious problem. 29%, 25%, 26% is a smaller problem too that would greatly benefit from a light touch of smoothing.
  • 3: It is hard to tell, should 2 & 3 goals peak that high? 4 & 5 goals probably should not be the same expected frequency.

The data is in excel. I am not looking for best practise, just some simple methods that would scale well based on the number of sets. It would be nice if it was a little less hacky / tacky than "three median/mean methods."

Obviously a lot of interference might make pretty graphs [not the aim] but is not statistically sound and would not hold up in the future. Meddling with expected frequencies feels intuitively bad practise, and could be frowned upon. I respect that if that is community consensus, as it is now without smoothing data, the expected frequencies work BUT are worse than actually using the mean. This should not be case for soccer but maybe the 7% tail above 5.5 is not that bad. However I would like to try gently smoothing the data to see if I can improve things so that at least using frequencies is equals predictive power of using mean.

Edit: The objective would be to smooth each set [row] individually. The first row is merely there to illustrate the average distribution and that any smoothing should really maintain a positive skew. Each set is expected frequencies of a given teams performance for their next match against an average team. Whatever the smoothing, the aim would be that it is actually a better representation of the raw expected frequencies. The smoothing desired would be anything that could be achieved using excel functions / formulas [even if it requires 100+ helper columnns, not an issue, performance is the real inhibitor].

$\endgroup$
  • $\begingroup$ The question is unclear. It looks like you have 16,000 observations of a 12-dimensional measure, or 16K 12-bar graphs.. What kind of smoothing are you looking for? The representative sample would be like an average of each column, yes? $\endgroup$ – Karl Jan 11 '16 at 16:20
1
$\begingroup$

I don't think Smoothing is the right term for what you want. I think you want each row to look a little more like the first row; so just make a new row N equal to the average of row N and row 1. You can use a weighted average if you want to tune how much similarity is required. Then, a third sheet to re-normalize each row into a total of 100%.

This procedure will yield a new dataset that is partially the original, and partly 'smoothed' towards your expected value (row 1).

$\endgroup$
  • $\begingroup$ I do "want each row to look a little more like the first row" but each set did not happen by chance. I want a similar skewed shape but not to dilute team's expected performance [eg revert back to mean]. By "representation" in the edit I do not mean appearances, I mean a better prediction. Set 2 is ridiculous to predict 0% chance of 6 goals, yet 7 goals as 9%. A hammer [method] needs to hit that down & realistically smoosh some of expected frequency to 5, 6, 8 & 9 goals but respect that most of the percentage that oozes from 7 goals should go towards 6 goals as 9 goals is very unlikely. $\endgroup$ – Evad Jan 11 '16 at 19:08
  • $\begingroup$ You're looking for a complicated model based solution, but you don't have a model specified any more precisely than row 1. Mixing 40% of row 1 and 60% of row N will achieve the hammer effect you're after and preserve the characteristics of row N. $\endgroup$ – Karl Jan 12 '16 at 3:35
  • $\begingroup$ Added more example sets. I use the method you described automatically using varying weighting percentages for teams with a low number of matches as there is less confidence in the results. What if a team wins there first match 7-0 [even if there next match is 1-1? Normally distributing results around a fluke result like that is catastrophic however as the recent number of games increase, confidence in results increases hence a point is reached where combining them with league average or some other average say of new teams does more harm than good in predicting future results. $\endgroup$ – Evad Jan 12 '16 at 6:20
  • $\begingroup$ The aim to maintain some statistical integrity is that the mean / median / mode should still be pretty close if not the same as the set originally was. In set 2, predicting that the team will score 7 goals 9% of the time means against an average team [or similar strength] that they will win 7% and draw 2% of the time, this effectively places a handicap on their future opponents unjustly. Spreading that frequency out while not changing the overall mean of the set out increases the chance of opponents being predicted to win. $\endgroup$ – Evad Jan 12 '16 at 6:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.