If I have a team, with between $n_1$ and $n_2$ people per team, with results of team's head to head matchups, how would I be able to estimate each person's value?
Example data (I drew this up quickly, the actual one is many lines longer, with more players per team):
| Home | Away | Home Score | Away Score | Home Members | Away Members | | ------ | ------ | ---------- | ---------- | ------------- | ---------------- | | Team 1 | Team 2 | 4 | 2 | John, Joe | Mary, James | | Team 1 | Team 3 | 3 | 2 | Joe, Patricia | Mark, Paul, Ross | | Team 2 | Team 5 | 1 | 2 | Mary, Robert | Steve, Lisa | | Team 4 | Team 3 | 4 | 3 | Ashley | Mark, Matt | | Team 4 | Team 5 | 2 | 3 | Donna, Emily | Anne, Lisa |
For the example data, I want to be able to say something like
John is the best with a individual score of $s_1$ and
Mary is right behind him with a individual score of $s_2$ and
James is the worst with a score of $s_n$. Thus, any team replacing any person with
John will become better and and team replacing any person with
James will become worse.
I am assuming that the score is a combination of everyone's individual scores and teamwork will not come into play (or more realistically, teamwork is built into each player's individual score). I do not, however, want to say that because
James scores all the points on every team he is on, that he is worth the most. It may be that James is simply the person that scores the points by setup of internal team communication, but if he is not good at this, his score should be lower (ex. If I run this against a football league, the strikers shouldn't always be at the top, with the defenders, presumably not having scored many goals, at the bottom).
I would also want to be able to weight matches (ie. match 1 is a regular match that doesn't really matter, but match 5 is an elimination match that really matters, so if a player consistently under-preforms the elimination matches, they have a lower score and visa-versa).
I care about both predicting the outcome of any match I want and placing a score on an individual player, but if I had to choose, I would place getting a more accurate individual player score at a higher importance.
The weights should also be tied to winning. However, it should be fine to tie it to the score (any team that scores more is generally better and any team that scores less is generally worse). However, the one problem with this is given a game where both teams score a million points and tie, both values would go up like crazy. However, ultimately they tied. So while you could argue that it shows potential to win later, I'm not altogether convinced that the scores should go up that much for a tie, because it also should go down for having a million points scored against them (by the football example, even the striker should have his scored, as they are not contributing to the defense or are not good at it - we also make no difference between strikers and defenders in the data, so it would be impossible to differentiate them).
I was thinking of something like an ELO system, where each player has their own ELO, but I'm not sure if this is the best way to do it. The problem with ELO is even if I give each player their own ELO ($e_1$), then calculate the team elo ($\sum e_n$) in order to determine the prediction and ELO multiplier, the player's will never reach their true level. Let's say a new superstar (so he has low/base ELO) joins a good team (so the average ELO is much higher than him, but his true ELO should be much higher than everyone else's — $\hat e_s < e_\mu$ but $e_s > e_\mu$). If the team starts winning and the ELO gains get evenly divided, the superstar never gains ELO compared to his teammates, thus being consistently undervalued by this system. The Bradley-Terry model or TrueSkill may be better solutions, although I'm unfamiliar with the shortcomings of both of these; Bradley-Terry has 2 players per team and TrueSkill relies on constant team switches, so I'm unclear how well they extend to this.
This seems fairly trivial, but I can't seem to wrap my head around how to do it.