"Investigator intention" and thresholds/p-values I'm reading John Kruschke's "Doing Bayesian Data Analysis" slides, but actually have a question about his interpretation of t-tests and/or the whole null-hypothesis significance testing framework. He argues that p-values are ill-defined because they depend on the investigator's intentions. 
In particular, he gives an example (pages 3-6) of two labs that collect identical data sets comparing two treatments. One lab commits to collect data from 12 subjects (6 per condition), while the other collects data for a fixed duration, which also happens to yield 12 subjects. According to the slides, the critical $t$-value for $p<0.05$ differs between these two data collection schemes: $t_{\textrm{crit}}=2.33$ for the former, but $t_{\textrm{crit}}=2.45$ for the latter!
A blog post--which I now cannot find--suggested that the fixed-duration scenario has more degrees of freedom since they could have collected data from 11, 13, or any other number of subjects, while the fixed-N scenario, by definition, has $N=12$.
Could someone please explain to me:


*

*Why the critical value would differ between these conditions? 

*(Assuming it's an issue) How one would go about correcting/comparing for the effects of different stopping criteria?
I know that setting the stopping criteria based on significance (e.g., sample until $p<0.05$) can inflate the chances of a Type I error, but that doesn't seem to be going on here, since neither stopping rule depends on the outcome of the analysis. 
 A: I finally tracked down the paper associated with the slides: Kruschke (2010), also available directly from the author (via CiteSeerX) here, since the journal is not widely carried. The explanation is a little bit prosaic, but I'm still not sure I buy it.
In the fixed-N case, the critical $t$-value is computed as follows: $2N$ samples are randomly drawn from the (same) population and a $t$-value is calculated. This process is repeated many times to build up a null distribution. Finally, $t_{crit}$ is set to be the 95th percentile of that distribution.
For the fixed duration case, he assumes that subjects arrive at a mean rate $\lambda$. The null distribution is constructed by repeating two steps. In the first step, the number of subjects for each condition $N_1$ and $N_2$ is drawn from a possion distribution with parameter $\lambda$. Next, $N_1$ and $N_2$ random draws from the population are used to calculate a $t$-value. This is repeated many times, and $t_{crit}$ is set to be the 95th percentile of that distribution.
This seems a little...cheeky...to me. As I understand it, there isn't a single $t$-distribution; instead it's a family of distributions, with a shape partly determined by the degrees-of-freedom parameter. For the fixed-$N$ condition, there are $N$ subjects per group and the appropriate $t$-value for an unpaired t-test is the one with $2N-2$ degrees of freedom, which is presumably what his simulation reproduces.  
In the other condition, it seems like the "$t$"-like distribution is actually a combination of samples from many different $t$-distributions, depending on the specific draws. By setting $\lambda=N$, one could get the average degrees of freedom to equal $2N-N$, but that's not quite enough. For example, the average of the $t$-distributions for $\nu=1$ and $\nu=5$ doesn't seem to be the $t$-distribution with 3 degrees of freedom.
In summary:


*

*The author was generating $t_{crit}$ by simulation, instead of just calculating them from the CDF.

*The way the author simulated the fixed-duration scenario seems like it might fatten up the tails of the corresponding $t$-distribution.

*I remain unconvinced that this is actually a problem, but would be happy to read/upvote/accept answers if anyone thinks otherwise.

A: Here's some more info: http://doingbayesiandataanalysis.blogspot.com/2012/07/sampling-distributions-of-t-when.html 
A more complete discussion is provided here: http://www.indiana.edu/~kruschke/BEST/  That article considers p values for stopping at threshold N, stopping at threshold duration, and stopping at threshold t value.
