3
$\begingroup$

Crosspost from Chess.SE.

TL;DR

I feel that ratings on a chess website are worth differently depending on the time of the day, and I want to validate the claim. I have a complete history of games played last several years. How should I formally state the hypothesis? What methods can be used to confirm or reject it?

Rating system

Various online chess sites use Elo or Glicko rating system. Briefly, each player has a numerical rating. The greater the difference in ratings of two players, the higher the probability of the higher-rated to win. The winner gains some rating points, and the gain depends on the rating of his opponent (win against the higher-rated opponent gives higher gain). Same for the loser. Exact formulas are available on Wiki pages by the link above, but they are not directly relevant.

Regarding the magnitude, here's the rating distribution for bullet (super-fast) chess at lichess.org: 600, 1175, 1450, 1775, 2800 for percentiles 0, 25, 50, 75, 100. Average rating gain after a game of two equally rated opponents is about 5 points.

Story

Once I noticed that my average performance varies depending on the time of the day: e.g. at the morning I can lose 50 rating points after a dozen of games, but I will easily regain it at the evening. One possible explanation is that my brain may work worse at the morning because of sleepyness or something. But can the reason be not me but my opponents?

Imagine two countries, A and B, in different time zones. Players from A and from B tend to play chess at the evening; let it be 00:00 UTC and 12:00 UTC respectively. And, for some reason, players from A are generally stronger than players from B. In the extreme example players from different countries never meet each other, so their ratings establish independently. So even if two players have similar strength, the player from B will have higher rating (because he is stronger among his population).

In this setting, if I play at 00:00 UTC I will meet opponents from A, and if I play at 12:00 UTC I will meet opponents from B. Players of my rating will have different absolute strength in these cases.

Formalization

I would like to formalize the claim and to find a method to validate it. Here's the best I could think of:

Assume that all players have some absolute strength, measured in the same units as the rating1. The outcome of each game is determined with probability according to Glicko formulas with regard to players' absolute strengths. Such system is stable enough, i.e. the rating of each player always fluctuates near its expectation. However, since the graph is not complete, the rating doesn't necessarily equal the absolute strength.

Let $R_n(x, t)$ be the random variable — the rating of a player of absolute strength $x$ after playing $n$ games at time $t$ o'clock UTC, and $R(x, t) = E\lim\limits_{n \to \infty} R_n(x, t) / n$ — the expectation of player's rating at the infinity.

Hypothesis: $R(x, t) = R(x)$, that is, $R(x, t)$ does not depend on $t$.

Question 1: Is this wording good enough for a formal hypothesis?

1 The absolute strength might be defined as the expectation of one's rating if every pair of players meets infinite number of times. However, I'm not sure whether this expectation depends on the order of games played so abstain from defining it this way.

Methods

I tried developing methods that can help rejecting the hypothesis; hardly any of them could confirm it.

  • Clustering. We build a multigraph with players as vertices and played games as edges. Then we cluster it somehow: either with some well-known algorithms, or ad-hoc like, say, dividing the clock into 24 hours, computing for each player the frequent hour of play and putting him there. Next we use the approach from this answer: assume that ratings of each cluster are shifted by a constant $\theta_{cluster}$ and estimate these constants with maximum likelihood or something. More elaborated approach, as described in the same answer, is to assume that rating shift itself depends on rating, so is a function; these functions also seem estimatable.

    However, there is little hope that clustering works. Unsupervised approach may fail since the graph seems too tightly connected. Clock-wise approach does not consider the fact that players may tend to play more than once a day (e.g. at the morning and at the evening).

  • Ordering. We somehow order all players in a circle such that in general everyone plays more games with neighbours than with distant fellows. Then we may introduce parameters similar to the previous approach: for example, $\theta_s$ is the rating shift for the $s$-th player in a row. Add some regularity ($|\theta_s - \theta_{s+1}| < \varepsilon$) and use MLE once again. Pitfalls are similar: no guarantee that reasonable circular ordering even exists. And if it does, it can be hard to compute.

  • Person-wise analysis. Look at the history of a certain player. If average gain is greater at some point hour, then likely rating points worth less at that point. However, if we take the average gain for all players... we get zero, since Glicko is (almost) zero-sum. So, sounds good, doesn't immediately work.

Question 2: Could you help me formulate the hypothesis and suggest any methods to validate it?

$\endgroup$
1
  • $\begingroup$ wait but if you say win more matches in the morning and lose more matches in the evening... then what about those people also playing in the evening: the matches you lost, someone else won...? i remember reading a similar post before for weekend vs weekday play in csgo (or some other game like perhaps a moba like LoL or dota) like someone said e had a higher win rate on weekdays as compared to weekends $\endgroup$
    – BCLC
    Jan 5 at 14:18

1 Answer 1

1
+50
$\begingroup$

While this might seem like a complicated problem, you can test this hypothesis in a pretty straightforward way using logistic regression. I'm assuming you do have access to the ratings of each player you face, and your own rating.

First, prepare your data. You'll need columns encoding the difference in ratings between yourself and your opponent for each match (let's call this $\delta$), the outcome (win or loss), and the time of day. Time of day should be binned. The size of the bins will depend on the data you have, but bins of 3 hours each sounds like a reasonable starting point.

Then, fit your baseline logistic regression model to this data:

$$ \text{Log-odds(Win)} = \text{Intercept} + \text{Slope} \times \delta $$

This means the log-odds of you winning a match depends on a) an intercept, which represents the log-odds when playing an opponent of the same ratings as you ($\delta = 0$), and the slope represents the change in odds as $\delta$ changes. This model is the null hypothesis: nothing is affected by time of day.

Next, fit the alternative model:

$$ \text{Log-odds(Win)} = \text{Intercept}_{\text{Time}} + \text{Slope} \times \delta $$

Here, the intercept term is estimated separately for each time bin, meaning that your chances of beating an opponent who has the same official rating as you can vary throughout the day - there is a main effect of time.

Finally, you can compare the two models. If the alternative model fits the data significantly better than the null model, you can reject the null hypothesis, and conclude that your hypothesis is correct.

In R code, this analysis would be something like the following:

null_model = glm(win ~ delta, data = your_data, family = binomial)
alt_model = glm(win ~ delta + time_window, data = your_data, family = binomial)
anova(null_model, alt_model)
$\endgroup$
5
  • 3
    $\begingroup$ OP should use a spline to expand the effect of time out into a non-linear function. In fact, they should use a cyclic cubic spline since there should in theory be a similar odds of losing just before and just after midnight. Additionally, OP should adjust their esitmate based on opponent's elo. $\endgroup$ Jan 6 at 2:29
  • $\begingroup$ Eoin and @DemetriPananos , i know i set the bounty but does this post have a point actually? every game the OP won at a certain time someone else won. assuming everyone is in the same time zone, I don't see how this is going to mean much. unless somehow time of day playing is really specific to the OP or something. $\endgroup$
    – BCLC
    Jan 6 at 8:30
  • 1
    $\begingroup$ I think the point is basically that (assuming the OP is in the UK) when they play in the middle of the night, they might be playing against someone in Australia, and if if they lose those matches more often that daytime matches against players of nominally the same ratings, it may be that Australian players are better than UK players with the same rating. $\endgroup$
    – Eoin
    Jan 6 at 9:09
  • 1
    $\begingroup$ Or, in other words, you can't assume everyone is in the same timezone. @DemetriPananos: Definitely, but let's start simple! $\endgroup$
    – Eoin
    Jan 6 at 9:13
  • $\begingroup$ Eoin ' you can't assume everyone is in the same timezone' --> ah yeah i guess. thanks! $\endgroup$
    – BCLC
    Jan 6 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.