Find a realistic distribution scenario for requests - Cross Validated most recent 30 from stats.stackexchange.com 2019-06-17T15:01:41Z https://stats.stackexchange.com/feeds/question/362207 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/362207 -1 Find a realistic distribution scenario for requests Lonzak https://stats.stackexchange.com/users/217622 2018-08-14T16:14:37Z 2018-08-15T09:29:00Z <p>For a loadtest I have to figure meaningful numbers for the maximum number of requests per hour and minute. The only thing I have is the number of requests per working day (which is a timeframe of 14 hours).</p> <p>The easiest (but most inaccurate) would be just to take the average but I think it follows a normal distribution. Here is what I have:</p> <p>There are people generating requests over the day from 06:00 (am) - 20:00 (pm) - afterwards the system is taken offline. They a generating an overall load of 1400 requests each day. So my question is: <strong>How could those requests be distributed over the day?</strong> <strong>I have something like this in mind</strong> (this is just an (unrelated) example but it should give an indication what I would image):</p> <p><a href="https://i.stack.imgur.com/IvocD.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IvocD.png" alt="enter image description here"></a></p> <p>I think the red curve comes the closest to my scenario since few people start at 6:00 or work until 20:00. The workload will increase at 9:00 and flat out at 17:00 => So like a normal Gaussian distribution...</p> <p><strong>How can that scenario be mapped into real data? I am especially interested in the peak number of requests during the day. So what could be a potential maximum number of requests per hour/minute?</strong> I know (as indicated by several graphs) that there are several potential distributions. (But the sum should always be 1400 in the end)</p> <p>I currently use excel to visualize the numbers/diagrams.</p> <p><strong>Update</strong>: </p> <ol> <li>Please note that the image is just an example which I highjacked to illustrate what I have in mind</li> <li>I do not have the actual data - only the daily number of 1400. If I had the actual timestamp of each request I wouldn't need to ask this question to figure out a realistic distribution scenario.</li> <li>I can not measure since it is historical data and this is also not an exam question ;-)</li> </ol> <p><strong>=> So the overall question is: What could be a possible distribution scenario to spread the requests over the day? I assumed that they followed a normal distribution and asked how that could look like...</strong></p> https://stats.stackexchange.com/questions/362207/-/362295#362295 2 Answer by BruceET for Find a realistic distribution scenario for requests BruceET https://stats.stackexchange.com/users/85665 2018-08-15T09:06:44Z 2018-08-15T09:29:00Z <p>Reading your comments I have learned a little more about your situation and @ERT makes useful suggestions. That prompts me to <em>speculate</em> how you might approach this problem by looking at real data.</p> <p>Suppose you have data for 200 days and you want to start your investigation by looking at the one busy hour between 13:00 and 14:00. By looking at the historical data you might find that the <em>average</em> number of requests during that hour has been 550. </p> <p><strong>Poisson model:</strong> If you assume that calls arrive randomly at the rate of 550 per hour at that time of day, then you might say that the number $X$ of calls per hour mid-day is distributed $\mathsf{Pois}(\lambda = 550).$ While you can't really find a realistic maximum and minimum that you will ever see during one mid-day hour, it might be useful to get an interval estimate designed to include requests during 90% of such hours. </p> <p><strong>Interval estimate for requests in a mid-day hour:</strong> If the Poisson model is right, you can cut off the bottom and top 5% of probability from $\mathsf{Pois}(550),$ which gives the interval $[512, 589].$ Then you could plan to have staff or bandwidth (or whatever) available between 13:00 and 15:00 each day to accommodate about 590 requests in a timely fashion. (I used the computations below in R statistical software, where <code>qpois</code> is a Poisson inverse CDF (quantile function) and <code>ppois</code> is a Poisson CDF.</p> <pre><code>qpois(c(.05,.95), 550)  512 589 diff(ppois(c(511, 589), 550))  0.9036664 </code></pre> <p><strong>Interval for a mid-day five-minuted period:</strong> I'm not sure I would push the Poisson model down the the one-minute level, but the rate of calls during a mid-day five-minute period would be $550/12 \approx 46$ and you could use a similar method to get an interval estimate that would express the number of requests arriving within 90% of typical five-minute mid-day time periods: $[35, 57].$ (Because of the discreteness of the Poisson distribution it is not possible to get an interval that contains <em>exactly</em> 90% of the probability--91% is close.)</p> <pre><code>qpois(c(.05,.95), 46)  35 57 diff(ppois(c(34, 57), 46))  0.9109021 </code></pre> <p>Similar analyses could be done for other times of day when the arrival rate of requests is smaller than at mid-day.</p> <p>I hope something like this approach will get you almost the kinds of answers you are looking for.</p>