How to approximate the point a sequence is converging to?

Question

I have created a poker solver as part of my Master's Thesis. This solver uses Counterfactual Regret Minimization (CFR) to compute a Nash Equilibrium of Hold'em or Omaha Poker. The solver uses existing solutions to resolve solutions starting at a certain game node in the tree, but that's besides the point.

It is known that the average strategies of players (the average of the strategies over all iterations) who use CFR converge to a Nash Equilibrium. Consequently, the average estimated payoffs of these players converge as well.

I am trying to approximate the Nash Equilibrium by converging up to the point that the average payoff of players in the approximated equilibrium is within X (big blinds) of the actual Nash Equilibrium. The challenge is that the payoff at the actual Nash Equilibrium is unknown.

Is there a way to use statistics in order to estimate within what range the point of convergence lies? I have been searching for quite a while (days-weeks), but haven't been able to find anything.

The question in a more general sense:

It's is known that points of a sequence X = (x1, x2, x3 , ... ,xN , ...) are randomly selected so that the sequence will converge to a point eventually. Based on the observations x1 to xN , what can we say about the value of the point it is expected to converge to?

The paper "Assessing Convergence of Markov Chain Monte Carlo Algorithms" describes some statistical methods and limitations to assess whether Markov Chains geometrically converge. Not entirely relevant to the question, but I haven't been able to find much more. https://www2.stat.duke.edu/~scs/Courses/Stat376/Papers/ConvergeDiagnostics/brooks97assessing.pdf)

The following question is closely related to it, but has not been answered https://stackoverflow.com/questions/77657691/algorithmic-game-theory-poker-cfr-and-the-approximation-distance

One of the things I thought would be a good indicator of convergence is a visual indications of the sequence X. Any conclusions drawn from looking at a plot of this sequence are of course subjective, so it's not sufficient for the purposes of a Master's thesis.

Answer 1

I figured out the answer to my question.

The difference in payoff of a player at strategy profile (s_1, s_2) and the Nash equilibrium is not calculated directly.

Let u_1(s_1, s_2) be the expected payoff for player 1 in a strategy profile. Let u_1(s_1, B) be the expected payoff of player 1 against the best response. Let D be the difference between these expected payoffs.

So, we have a minimum of the payoff of the strategy in the Nash equilibrium: u_1(s_1, B). We know that this minimum is at most D away from the estimated payoff at equilibrium, because B is not an optimal strategy, so player 1 should be able to get a higher estimated payoff against B. So, the estimated payoff for player 1 at equilibrium is at most u_1(s_1, B) + D.

The payoff of a best response can be directly computed through the regrets of players in the iterations before.

If any of you are interested in the theoretical approach to assess the convergence of a sequence, the paper "Convergence Tests for the Fission Source Distribution in MCNP5" may give some inspiration. I believe it is a similar problem.