I am creating a testing dashboard for displaying the results of our website testing. We run A/B tests and Multivariate tests. My testing code is sound (the gathering of the raw conversion data), but now I am working on finishing out our results reporting admin dashboard. (before we used a very old version of JMP to analyze our results but that was getting tedious so we/I decided to code up an admin dashboard with R)
Here is an example set of our data. (for an A/B test that we ran)
>a=c(1.3571428571429,1.3941176470588,1.275,1.2408759124088,1.5842696629213, 1.5537634408602,1.4590909090909,1.4018691588785,1.495145631068,1.1794871794872, 1.5114503816794,1.2569832402235,1.5336538461538) >b=c(1.4891304347826,1.4656084656085,1.44,1.5190839694656,1.6084656084656, 1.4166666666667,1.5665024630542,1.2533333333333,1.4081632653061,1.6173913043478, 1.468253968254,1.5392670157068,1.5081081081081)
And with that data I am running the following R code to calculate the p-value / statistical significance.
>t.test(a, b, paired=TRUE)
Now after a week or two we would like to know how long we would need to continue to run the test before we would get to a given level of statistical significance, given the current data that we have collected.
n.ttest but I have since been told that this is more meant for power analysis.
So my question is what R function should I use to get this information? Also on a related note I would be interested to know if there is any other calculations I should be running on our test data that we need that I could also add to our testing dashboard.