Poker winrate statistical test In the game of poker, how do we test $H_o : \textrm{winrate} > 0$? Or how can we test $H_o : \textrm{winrate} = x$ for $x \in \mathbb{R}$? Winrate might be measured as the population $ profit per hand.
If we think of the average player's profit graph as something like a Wiener process then something that looks very good will often be within the variance interval at $t$ made by $Profit_t \sim N(0,t)$. 
 A: Some of the following information is in my book, "The Math of Hold'em." This is aimed at poker players not mathematicians, though.
First, let me clarify the context.
The vast majority of players don't play very much. In an online environment, they deposit something like $\$20$ or $\$50$, and play a short time until it is gone, often playing a mixture of different games, some tournaments and some ring games. In live games, you have many people who play only for a couple of evenings. You don't have much data on individual players like this. 
If you want to ask how poker players do as a group, they play each other so they lose the rake on average. 
The percentage of hands won is not the right statistic to use. Many professional players at a $6$-handed table will win far less than $1/6$ of the hands. The goal is not to win as many hands as possible, but to profit. A common mistake of casual players is to play too many hands, which is like buying lottery tickets. This decreases the percentage of hands won by the other players, but makes them win enough extra money when they do win to make up for the lower winning percentage. So, win rate does not refer to the percentage of hands won, but to the profit per unit of play.
The usual time when it makes sense to ask about win rates is when you have a serious player who has collected a large amount of data on his (or her, but it is overwhelmingly male) own play. There are programs such as HEM and PokerTracker which collect this data automatically as the player plays. There are also third party websites like Sharkscope.com which try to collect data on players. Some of these do not have uniform coverage and may be biased, e.g., if I see Sharkscope has missed one of my tournaments, I can ask them to look it up and add it to their records, and I may be more likely to do this with tournaments I have won than those where I was knocked out early.

Ring games: Hands are not independent. Even if players do not act emotionally to past wins and losses, and do not learn from past hands, there are still several failures of independence. 


*

*In Hold'em or Omaha where there are blinds instead of a forced bring-in in stud, your position advances towards the blinds on each hand until you pay them. In theory, you should only play hands which are profitable outside the blinds, but your gains outside the blinds are balanced by the losses from paying the blinds. You should play fewer hands when you are in early positions and have the disadvantage of acting before several other players. So, players win signficantly different amounts from different positions. A player might average $0.30$ big blinds/hand from the "button" or best position, while winning only $0.05$ big blinds/hand from $5$ positions off the button, and while losing $0.40$ big blinds/hand from the big blind itself. The variances are also different. You can try to compensate for this in several ways. You can group hands into orbits, you can separate the hands by position, or you could ignore this effect. Each of these has its merits and disadvantages.

*The table composition, and even the number of players at the table, is not randomized after each hand. A loose player may tend to transfer money from players on his right to players on his left, whether he is a net winner or loser. If you have a loose player on your right this hand, you might have that player on your right for the next $200$ hands. Some players are much stronger when there are some open seats at the table, which is common. Poker players recognize that some situations are particularly profitable, and some are unprofitable. If you only look at the variance by hand, you will underestimate the total variance due to the luck from finding good tables.

*In no limit games, the variance depends on the effective stack depth, and to some extent in practice on the absolute stack depth. Unlike in the movies, you can only lose what you have in front of you, and only if your opponents have at least as much. If you accumulate a large stack, your variance per hand increases in square blinds/hand, although it becomes less common to lose your entire stack or to double up in one hand. Even with a large amount of data on all hands, I still have poor estimates of my win rate and variance when I have several times the maximum buy-in at the table, since these hands are less common and the variance is larger. 
That said, people have a lot of hands, and they use normal approximations ignoring the difficulties. The standard deviation is estimated by tracking software, and it is usually about $17$ big bets ($34$ big blinds) per $100$ hands for serious players in shorthanded online limit hold'em games. Standard deviations for no limit hold'em are more sensitive to playing style and the size of the minimum and maximum buy-ins allowed, but a value of $90$ big blinds per $100$ hands may be typical for some players who buy in for $100$ big blinds. 

Tournament play: There are some single table tournaments with at most $10$ places, and a maximum prize which is only a few times the buy-in. There are some multitable tournaments which can have thousands or tens of thousands of players, with a significant percentage of the prize pool awarded to the top $3$ places. 
Single table tournaments (STTs) run more often, and it is much easier to estimate win rates and variance for these tournaments. For a $50-30-20$ prize structure, the standard deviation per tournament is about $1.5$ buy-ins for a serious player. This is not particularly sensitive to playing style or win rate. Some STT players complete over $20$ tournaments per hour, so there is a lot of data on some players, enough that some players know their returns on investment (ROIs) within a percent or two. Normal approximations seem appropriate, and everyone uses them.
A significant proportion of the luck in no limit tournaments occurs after players get all-in, when all hands are exposed and the remaining community cards are dealt to determine the winner. Personal tracking programs use variance reduction on these hands in single table tournaments. They use tournament equity models such as the ICM to make an unbiased estimate of luck which they subtract from the player's results. This reduces roughly half of the variance away, and perhaps $70\%$ of the variance in "super turbos." Instead of requiring about $1000$ tournaments to get a $95\%$ confidence interval of $\pm 10\%$ ROI, it only takes $300$ to $500$ STTs.
Multitable tournaments (MTTs) are much more difficult to analyze. They have different numbers of entrants and difference prize structures from each other, there are more rule variations such as rebuys and add-ons, and there is less data. It is known that the standard deviation increases with the size of the tournament and is much more sensitive to the playing style of each player. It is very hard to get an accurate estimate on win rates from results alone, since an important question is how well the player performs near or at the final table, and the overwhelming majority of tournaments where the player is knocked out early provide little information about this.
There are some unscheduled MTTs of fixed sizes which start when they are full. These behave more like STTs. For example, $180$ player tournaments often run on one major site. The standard deviation is about $5$ buy-ins per tournament for a player who finishes in all places with equal probability, but some winning players have standard deviations of about $6-9$ buy-ins per tournament. 
Winning MTT players have larger ROIs than winning STT players, and they do not need to estimate them as accurately as STT players do to make good decisions about which tournaments to enter. However, the difficulties are too great for most players. Most serious players of scheduled MTTs do not know their ROIs accurately at all. There is a lot of room for progress estimating win rates in MTTs. 
One of the biggest MTTs each year is the World Series of Poker main event. There are thousands of entrants, many of whom are casual players, so it is supposed to be a very profitable tournament for MTT experts (although some unknown professionals are portrayed as novices, and some famous casual players are portrayed as professionals on TV). The standard deviation could be over $20$ buy-ins for the experts, and the WSOP main events have only reached this size in the past decade. So, no one has enough data to estimate ROIs very accurately in the WSOP main event.
