How do you tell whether good performances come in streaks?

Question

I solve Rubik's cubes as a hobby. I record the time it took me to solve the cube using some software, and so now I have data from thousands of solves. The data is basically a long list of numbers representing the time each sequential solve took (e.g. 22.11, 20.66, 21.00, 18.74, ...)

The time it takes me to solve the cube naturally varies somewhat from solve to solve, so there are good solves and bad solves.

I want to know whether I "get hot" - whether the good solves come in streaks. For example, if I've just had a few consecutive good solves, is it more likely that my next solve will be good?

What sort of analysis would be appropriate? I can think of a few specific things to do, for example treating the solves as a Markov process and seeing how well one solve predicts the next and comparing to random data, seeing how long the longest streaks of consecutive solves below the median for the last 100 are and comparing to what would be expected in random data, etc. I am not sure how insightful these tests would be, and wonder whether there are some well-developed approaches to this sort of problem.

Answer 1

The Wald-Wolfowitz Runs Test seems to be a possible candidate, where a "run" is what you called a "streak". It requires dichotomous data, so you'd have to label each solve as "bad" vs. "good" according to some threshold - like the median time as you suggested. The null hypothesis is that "good" and "bad" solves alternate randomly. A one-sided alternative hypothesis corresponding to your intuition is that "good" solves clump together in long streaks, implying that there are fewer runs than expected with random data. Test statistic is the number of runs. In R:

> N      <- 200                          # number of solves
> DV     <- round(runif(N, 15, 30), 1)   # simulate some uniform data
> thresh <- median(DV)                   # threshold for binary classification

# do the binary classification
> DVfac <- cut(DV, breaks=c(-Inf, thresh, Inf), labels=c("good", "bad"))
> Nj    <- table(DVfac)                  # number of "good" and "bad" solves
> n1    <- Nj[1]                         # number of "good" solves
> n2    <- Nj[2]                         # number of "bad" solves
> (runs <- rle(as.character(DVfac)))     # analysis of runs
Run Length Encoding
lengths: int [1:92] 2 1 2 4 1 4 3 4 2 5 ...
values : chr [1:92] "bad" "good" "bad" "good" "bad" "good" "bad" ...

> (nRuns <- length(runs$lengths))        # test statistic: observed number of runs
[1] 92

# theoretical maximum of runs for given n1, n2
> (rMax <- ifelse(n1 == n2, N, 2*min(n1, n2) + 1))
199 

When you only have a few observations, you can calculate the exact probabilities for each number of runs under the null hypothesis. Otherwise, the distribution of "number of runs" can be approximated by a standard normal distribution.

> (muR  <- 1 + ((2*n1*n2) / N))                     # expected value
100.99 

> varR  <- (2*n1*n2*(2*n1*n2 - N)) / (N^2 * (N-1))  # theoretical variance
> rZ    <- (nRuns-muR) / sqrt(varR)                 # z-score
> (pVal <- pnorm(rZ, mean=0, sd=1))                 # one-sided p-value
0.1012055

The p-value is for the one-sided alternative hypothesis that "good" solves come in streaks.

Answer 2

A few thoughts:

• Plot the distribution of times. My guess is that they will be positively skewed, such that some solution times are really slow. In that case you might want to consider a log or some other transformation of solution times.

• Create a scatter plot of trial on the x axis and solution time (or log solution time on the y-axis). This should give you an intuitive understanding of the data. It may also reveal other kinds of trends besides the "hot streak".

• Consider whether there is a learning effect over time. With most puzzles, you get quicker with practice. The plot should help to reveal whether this is the case. Such an effect is different to a "hot streak" effect. It will lead to correlation between trials because when you are first learning, slow trials will co-occur with other slow trials, and as you get more experienced, faster trials will co-occur with faster trials.

• Consider your conceptual definition of "hot streaks". For example, does it only apply to trials that are proximate in time or is about proximity of order. Say you solved the cube quickly on Tuesday, and then had a break and on the next Friday you solved it quickly. Is this a hot streak, or does it only count if you do it on the same day?

• Are there other effects that might be distinct from a hot streak effect? E.g., time of day that you solve the puzzle (e.g., fatigue), degree to which you are actually trying hard? etc.

• Once the alternative systematic effects have been understood, you could develop a model that includes as many of them as possible. You could plot the residual on the y axis and trial on the x-axis. Then you could see whether there are auto-correlations in the residuals in the model. This auto-correlation would provide some evidence of hot streaks. However, an alternative interpretation is that there is some other systematic effect that you have not excluded.

Answer 3

Calculate correlogram for your process. If your process is gaussian (by the looks of your sample it is) you can establish lower/upper bounds (B) and check if the correlations at given lag are significant. Positive autocorrelation at lag 1 would indicate existence of "streaks of luck".

Answer 4

Before looking at any formal tests, I would suggest you begin by looking at an autocorrelation plot for your data, to see if the solve-time is correlated with previous solve-times at past lag values. If you get "hot streaks" in your performance then this should manifest as positive autocorrelation over at least one lag, and possibly a few more. You might be able to model your solve-times with some kind of time-series process with one or more auto-correlation terms, and then formally test for "hot streaks" by testing for non-zero autocorrelation in the process.

Answer 5

I think there are several issues here the most pertinent being that presumably the gap between individual solves isn't the same. You may have to define a "maximum gap" so that you can treat the data as if it is sequential, however that will probably create a whole bunch of partitions in your data. I.e. instead of having one long sequence of solve times, you'd have a whole bunch of separate sequences.

I think what you're asking essential is if the the present solve time is conditionally independent to the one before it. There are myriad ways to test for that specific condition. Here is a few Bayesian approaches:

https://www.bnlearn.com/examples/ci.test/