Skip to main content
Cross Validated

Return to Question

added 36 characters in body; edited title
Source Link
cardinal
cardinal
  • 27.3k
  • 8
  • 105
  • 140

Why is running split tests UNTILuntil statistically significant a "Bad Thing""bad thing"? (orOr is it?)

I read this article: http://www.evanmiller.org/how-not-to-run-an-ab-test.htmlthis article on "how not to run an A/B test".

And I still don't understand what exactly the author's reasoning is. Can someone dumb it down for me?

I think what it might be saying is that reading the results of my split tests over time misleads me. I want to be able to understand this well enough that I can explain it to others, though.

Any help?

Why is running split tests UNTIL statistically significant a "Bad Thing"? (or is it?)

I read this article: http://www.evanmiller.org/how-not-to-run-an-ab-test.html

And I still don't understand what exactly the author's reasoning is. Can someone dumb it down for me?

I think what it might be saying is that reading the results of my split tests over time misleads me. I want to be able to understand this well enough that I can explain it to others though.

Any help?

Why is running split tests until statistically significant a "bad thing"? (Or is it?)

I read this article on "how not to run an A/B test".

And I still don't understand what exactly the author's reasoning is. Can someone dumb it down for me?

I think what it might be saying is that reading the results of my split tests over time misleads me. I want to be able to understand this well enough that I can explain it to others, though.

Any help?

Tweeted twitter.com/#!/StackStats/status/155339549525016576
Source Link
Justin Bozonier
Justin Bozonier
  • 1.2k
  • 2
  • 12
  • 24

Why is running split tests UNTIL statistically significant a "Bad Thing"? (or is it?)

I read this article: http://www.evanmiller.org/how-not-to-run-an-ab-test.html

And I still don't understand what exactly the author's reasoning is. Can someone dumb it down for me?

I think what it might be saying is that reading the results of my split tests over time misleads me. I want to be able to understand this well enough that I can explain it to others though.

Any help?