Odds of winning 2 trials in a row There are 8 people playing poker.
So, the odds of winning the entire round = 1/8
2 rounds are played, and Bill wins both rounds.
What are the odds this was random?  (Hypothesis test?)
NullH = Bill has no added skill.  (Got lucky)
AltH = Bill has skill.
p = .13  = 1/8
q = .87 = 7/8
n = 2
SD = sqrt(pq/n) = .23
actual (p-hat) = 1
z = 3.74
p-value = 0%

Conclusion: Odds of winning 2 out of 2 rounds randomly is unlikely.
Reject null hypothesis.  Bill has skill.
Is this right?
Thanks!!
 A: No, this is not right, because no observational context has been provided.  All the following scenarios are consistent with the information given:


*

*Two games will be played.  Beforehand, you hypothesize that Bill will win both.  Assuming the results are independent and Bill has 1/8 chance of winning, the chance of this occurring is 1/64 = about 1.5%.

*Again two games will be played.  Beforehand, you hypothesize that somebody will win both games.  Under the same assumptions the chance of this occurring is 1/64 for each player.  Because all eight possibilities (for the eight players) are mutually exclusive, the chances add, giving a probability of 1/8 = 12.5% for this outcome.

*This time an indefinite number of games will be played, with the series stopping when Bill wins two in a row.  The chance that the series does end is 100% and we observe that Bill won the last two games.

*Again an indefinite number of games will be played, with the series stopping when anybody wins two in a row.  The chance the series ends is 100% unless there is perfect anti-correlation among the winners.  Assuming all players have equal chances of winning, the chance that Bill is the one to end it is 1/8 = 12.5%.
Because neither a clear null hypothesis nor an alternative have been specified, this is not a well-defined hypothesis testing situation.  Therefore the idea of "p-value" is meaningless (and is unnecessary anyway).  In scenario #1 the low probability provides some evidence--not much--that Bill is not just winning independently at random.  Some alternative explanations include (a) some other players have essentially no chances of winning and Bill just got lucky; (b) the cards are not shuffled well between games, causing the winner of one to be the likely winner of the next, and Bill just got lucky in the first game; (c) in some games certain players have higher chances of winning not through skill but due to their position in the deal, and Bill happened to be in a good position in one or both games.
We cannot generally conclude that the winner of two successive games has "skill".  For instance, he might have a partner who is setting him up to win.  Or he might be the mark in a group of card sharks who are letting him win to get his confidence, etc.
A: What you are observing is a random variable with binomial distribution, where number of trials is 2 and success probability $1/8$. You then can calculate p-value directly, without resorting to normal approximation. Your statistic is then the number of successes, which under null hypothesis is distributed as $Bin(n,p)$, with $n=2$, and $p=1/8$. The p-value is then
$$P(\hat{X}\ge 2)=P(\hat{X}=2)={2\choose 2}\left(\frac{1}{8}\right)^2\left(1-\frac{1}{8}\right)^0=\left(\frac{1}{8}\right)^2=0.015625$$
So yes the null hypothesis should be rejected.
A: This a job for Bayes Theorem, not for null-hypothesis testing. You've given us some information, so we should now simply determine how much more or less consistent that information is with the hypothesis than with its negation & adjust our priors accordingly. Based on what you've told us I conclude that the likelihood "Bill is skilled" is low--somewhere in the neighborhood of 10% (that is, I'll bet he is "lucky" unless you offer me odds better than 9:1 against "skilled"). Here's why:


*

*Neither the failure to make hypotheses in advance of the 2 games, the lack of information about how many games have been played or will be played after the 2 in question, nor the possibility that someone is cheating, etc. really matter. Having been supplied no information bearing on the situation, we should assume an 8-player poker game drawn at random from the universe of such games being played. That deals w/ w/ all the "what ifs" & "could bes" that Whuber draws our attention to: b/c the proportion of games that are fixed, or involve 1 avg player competing w/ 7 imbeciles, or are played w/ unshuffled decks etc., is tiny in comparison to the proportion of games that are fair & involve "run of the mill players," you will just be making your life complicated if you assume this game involves anything other than a "normal game" (if you have a different sense of how the universe of 8-person games is populated, then modify this part of the analysis accordingly; I'm just trying to demonstrate how to think about this problem!).

*Based on billions of games & meticulous record-keeping, I put the likelihood at about 0.10 that any randomly selected player in a normal game of poker is a "skilled" (again, if you have a different sense of what the talent distribution looks like, substitute your own estimate here).

*In a normal game of poker, the likelihood that a skilled poker player will win any 2 consecutive hands against 7 randomly selected players is only a scintilla higher than the likelihood that an unskilled one will. Poker is a game of skill, yes, but variance is super high (if you think otherwise, fine, but in that case you definitely are not an experienced poker player). If you had said "Bill made 2 consecutive 40-foot jump shots," or "won 2 consecutive olympic marathons," in contrast, then my priors about how much better he is than the average baseketball player or the average marathon runner would shift much much more dramatically-- those are lower-variance indicators of those types of skill.

*Because the likelihood ratio for the hypothesis and the negation of the hypothesis is thus very close to 1, you won't do much better here than going with your priors. Again, mine is that there is 1/10 chance that Bill is skilled.
