Determining proper K value for Elo rating

For the question I previously posted here with a general question about elo ranking applied to a a bit weird sport. From the ranking POV it's a bit like marathon - you can have up to n players but - unlike i.e. chess - your final ranking is based on single performance on your part and final result is based on the number assigned to your single performance.

Is it important? Kind of.

I could represent this sport as n^2 - n pairs. This sucks a bit because it makes a lot more important for your elo ranking to win massively populated tournaments than to win several small. It may inflate one's ranking based on just one-time fantastic result that just happened to coincide with a massively populated run.

On the other hand one can take the average of all the results and make each tournament worth just like one tournament (one run after all). This however leads to small fluctuations of the ranking if the competitions are not really that common (Which is a case in my case). You can offset it by big K parameter but that might lead to very volatile ranking. Also it feels that 2 competitor tournament shouldn't be worth just as much 20 people tournament.

After trying both of the above approaches I settled for the latter but with variable K - each competition has K growing with the amount of competitors. So 1/3 of K is static and 2/3 depends on the number of competitors. This kind of looks OK (OKish maybe) but I still have my doubts.

And to the question: how reasonably verify that you've chosen correct K value? So far I just build a ranking, plotted it and looked for something I found weird. But that doesn't sound mathematical. Or maybe variable K is not good solution?

I'm aware I can apply suggestion from here: How to prove that Elo rating or Page ranking have a meaning for my set? but that's not completely what I'm asking about.

• Should i.e. go through predictions step by step and calculate sum of differences between expected values and real values? Would that make sense? I imagine I could devise genetic alghorithm that would work on the data and adjust formula for K based on the data at hand. Jun 21, 2016 at 11:43

I discussed briefly this solution with professional mathematician and he didn't discard the solution.

K depends on n the number of competitors: $$K = An^2 + Bn + C$$

Then I worked on my data set through various values and assumed that best chosen K should have minimal differences between expected result and actual result (makes sense, doesn't it?) data-set wide. It turned out that best polynomial on my data set had $$A=0$$ and $$C=0$$. This means that I actually could treat the whole tournament as round robin after all.

In my case the optimal value was $$B=42$$ (I required for the value K to be represented by natural numbers) which is nice but only coincident.

Considering each game of multiple players be a group of 1vs1 matches between all pairs of players is not the best option, and it leads to the problems you said.

If you're interested in use (more than in development), you should give a try to rankade, our ranking system (here's a comparison between most known ranking systems). Rankade, as a post-Elo ranking system, manage a multiplayer match as a single multiplayer match (and not as many fake 1-on-1 matches), having a variable $K$, depending on number of players (and many other factors, as playing frequency and group characteristics, and more).

• This is external and copyrighted system. I don't see how that answers my question. If you let everyone use it and provide some documentation on how it works internally I will be willing to look into it. Otherwise I'd rather go further into glicko systems or one published on Aligulac. Jul 5, 2016 at 9:30
• Rankade provides something that Glicko and Elo do not: a real multiplayer match structure. It solves your issue (and in my opinion it's an answer to your question, avoiding fake pairs and $K$ problem), and I wrote explicitly "If you're interested in use (more than in development)". If you do not, no problem, go for Glicko or other rating system (but I confirm the first part of my answer :) ). Jul 5, 2016 at 20:48