1
$\begingroup$

Suppose I want to use a (deep) neural network to output a probability that is normalised to sum to 1 within each of a potentially arbitrary number of groups.

For example, this might be something like attempting to project the percentage of a team's goals that a given player will score for his team during a given game.

The data might look something like this:

 FixtureId     TeamId      PlayerId    Actual % of Team Goals     Explanatory Variables
 1             1           1           0.8                        ...
 1             1           2           0.2                        ...
 1             1           3           0                          ...
 1             1           4           0                          ...
 1             2           5           1                          ...
 1             2           6           0                          ...
 1             2           7           0                          ...
 1             2           8           0                          ...
 2             1           1           0.5                        ...
 2             1           2           0.5                        ...
 2             1           3           0                          ...
 2             1           4           0                          ...
 2             2           5           0                          ...
 2             2           6           1                          ...
 2             2           7           0                          ...
 2             2           8           0                          ...
...

where we're attempting to model Actual % of Team Goals ~ Explanatory Variables.

The way that I look at this is we should probably have an arbitrary number being produced by some layer, with the output from each individual fixture and team combination being fed through a softmax layer, and then output.

(I think that it makes sense to use one row for each player here rather than one row per fixture/team given the potentially large number of explanatory variables/features.)

I guess I have a number of questions. Does any of this formulation make sense? How would a potential NN solution be achieved (especially as there's the potential for problems during training with batching, etc)? Are there any much more sensible ways to go about this?

I'm familiar with Keras, and know I can create my own loss function/activation function. But given there could be an arbitary number of fixtures/teams/players within each team, I'm not sure if that level of complexity is possible.

(I couldn't even find a name for this kind of problem - any help at this stage would be much appreciated.)

$\endgroup$
1
$\begingroup$

Suppose I want to use a (deep) neural network to output a probability that is normalised to sum to 1 within each of a potentially arbitrary number of groups.

Softmax activation has this property, and is pretty much the default choice for problems of this nature. Softmax generalizes the inverse logistic function to $k \ge 2$ categorical outcomes.

It is defined as

$$ f(\mathbf{x})=\frac{\exp(x_i)}{\sum_{i=1}^k \exp(x_i)} $$

and is clearly bounded between 0 and 1, and the sum of the elements of $f(\mathbf{x})$ must equal 1.

$\endgroup$
  • $\begingroup$ Right. But how would I manage to have this softmax (and thus the sum to 1 property maintained) across an arbitrary number of rows, rather than columns? Apologies if I am being dim. $\endgroup$ – cmcsorley17 Jul 16 '18 at 1:11
  • $\begingroup$ To go into a bit more detail...if I had the data in the format: FixtureId TeamId Player1%ofTeamGoals Player2%ofTeamGoals .... Player1ExpVars Player2ExpVars ... (where these are the columns) then I could fit e.g. Player1%ofTeamGoals, Player2%ofTeamGoals, ... PlayerP%ofTeamGoals ~ Player1ExpVars, Player2ExpVars ... PlayerPExpVars with a softmax output layer, and that would give me what I want. However, I want to be able to feed in the data as one player per row. Does that make sense? $\endgroup$ – cmcsorley17 Jul 16 '18 at 1:14
  • 1
    $\begingroup$ @cmcsorley17 Re-arranging your data so that the outcome sums to 1 along a specific axis seems to solve this problem. For example, a 3-tensor would work. $\endgroup$ – Sycorax Jul 16 '18 at 1:38
  • $\begingroup$ So if I'm understanding correctly...I would need to rearrange my data as a 3D-matrix M_ijk, where each i refers to some team/match combination, each j to a player, and each k to a variable (with the % of team goals parameter being one of them)? $\endgroup$ – cmcsorley17 Jul 18 '18 at 0:34
  • 1
    $\begingroup$ This is the basic idea. But note that you only need to re-arrange the data that stores the outcome, and probably the final layer of the network. $\endgroup$ – Sycorax Jul 18 '18 at 0:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.