Estimating better performing variation with unkown sample size Scenario: We are in the app business and trying to optimize for downloads in the app store. Things like the shown screenshots, the text, the icon etc. can be changed and will have an influence on the number of downloads.
Example:
Setup 1, downloads in a week: 100
Setup 2, downloads in a week: 110
I believe that this differs from normal A/B testing on websites, as we unfortunately do not know the sample size, i.e. how many people looked at the app profile. We only know how many decided to download and that the timeframe is the same, assuming that all others things stayed equal during the two time periods, maybe we can at least estimate the two sample sizes to be equal.
Still it should be possible to infer something from these very limited numbers. Possibly after estimating a few things about the sample size and the population in general. Another difference to the A/B testing is that the variations are carried out in sequence and not in parallel. Of course, at this point we can only assume that factors other than what we changed have stayed constant.
Can anyone give pointers as to how get the most out of this limited data? A rough estimate of which variation is better by which confidence interval is all we need
 A: If you only have measurements for two one-week-long sequential periods, there's probably not much you can do to get a signal from it--there's just too much noise between the two weeks.
However, if you can randomize the app profile between A and B more quickly, and measure the number of downloads each time, then you can still get a proper randomized design. That is, instead of having the data points
setup   time        downloads
A       12/1-12/7   100
B       12/8-12/14  110

you might have the data points
setup   time    downloads
A       12/1    10
A       12/2    8
B       12/3    11
A       12/4    3
A       12/5    7
A       12/6    12
B       12/7    8
B       12/8    5

You can then think about whether A or B points have a significantly higher number of downloads per day. This becomes more powerful the faster you can re-randomize the profile, since there will be less confounding noise on a minute-to-minute basis than day-to-day, and less day-to-day than week-to-week, etc. (As @whuber noted below in response to an earlier version, you basically need multiple observations for each profile in order to apply a probability model to the data.)
(Note that in this scenario your data are unlikely to be normally-distributed, so you should be careful with your hypothesis testing, perhaps using a bootstrap.)
If you can't re-randomize it fast enough, you will need a long time to get enough statistical power. You should calculate how many data points you expect to need beforehand based on the distribution of downloads-per-time-period that you currently have, so that you don't just keep running the experiment until you have a significant result and then stop immediately; this will bias your results.
