Let's say I have a website with certain number of visits per day and conversion events:
Date Visits Conversion_Events Conversion_% 12/1/14 3,369 179 5.3% 12/2/14 3,297 199 6.0% 12/3/14 3,355 405 12.1% 12/4/14 2,035 220 10.8% 12/5/14 4,834 207 4.3% Total 16,890 1,210 7.2%
These numbers fluctuate for multitude of reasons that are unknown to us (we can't control for geo location, demographics, etc., and assuming day-of-week, day-of-month, etc. don't make sense to control for). But we do know day-by-day data fluctuates around some mean. I'm interested in coming up with a plausible 95% confidence interval for tomorrow's (or any other day's) conversion based on historical data.
I can think of a few ways to achieve this:
(1) Look at totals only and use "standard error of proportion" formula: Sp = sqrt(p*(1-p)/n) = sqrt(7.2%*(1-7.2%)/16890). This means in this example 6.8%-7.6%. Intuitively, this seems to me to hide the fact that the day-by-day conversions fluctuate, so I'm concerned this loses some valuable information.
(2) Look at every date as a sample of the population, and calculate mean and stdev across samples. I.e. average() and stdev() on the array [5.3%,6.0%,12.1%,10.8%,4.3%]. Now, I get it that mathematically averaging proportions with varying sample sizes is not great, but intuitively assuming that sample sizes are very large it seems to me that this effect should be mitigated by the fact we are looking at the proportion summary statistic. Note that I gave an example of 5 days, but in reality, I can get a high number of "samples", so I can have a pretty good understanding of the distribution function of day-by-day proportions. This means the 95% confidence interval in this example is 0.8%-14.6%, which is quite big (but probably due to the fact that the numbers in this example are just made up).
(3) Calculate 95% confidence interval of Visits and of Conversion_Events, then take the low/high and high/low. I.e. Visits 95% confidence is 1435-5674, Conversion_Events is 61-361, so 61/5674=1.1% and 361/1435=25.2%. This makes some intuitive sense to me, though I suspect that simply dividing in such manner no longer means the outcome confidence is at 95%.
So, my questions:
a. Is my concern that method (1) is "hiding" some information justified?
b. Does method (2), which intuitively makes the most sense to me, make statistical sense?
c. What's the preferred approach (which I suspect is neither of the 3)?