How to predict and optimize a queue?

Question

I am trying to determine if I can model my system as a M/M/1 queue and if so do the numbers I get from it help me at all.

I can model my system like this:

System Description

A. I want to spec out the max number of clients I can have connect to my system. Each client can generate at max 40 messages/second that are placed in the buffer. If link B goes down it will buffer all messages. For the sake of my math I am going to assume that it is actually generating a fixed 40 messages/second.

B. I am specifying that this link should be able to go down for 3 days, but normal operation spec this as a network connection that all clients will have to communicate on. This network connection may also have traffic unrelated to my system on it. The link is fast enough that it isn't a bottle neck at all. I have found that it can easily pass 500,000 messages in less then 10 seconds.

C. Allows for the clients to dump any data as fast as link B will allow. This queue can be as big as needed.

D. This isn't really a link per say, block E pulls directly from the queue.

E, F, and G. This is where the fun comes. Initially every packet I received I placed a single SQL query to the database. I found that each query took about 5-15ms with a few taking 1ms and a few taking 600+ms. I don't have a model for this delay and suspect I will need one.

I realized that this delay was going to drastically hurt my systems performance so I started looking into how to improve this performance. I found that if I grouped multiple messaged from C together and did a larger SQL query I was able to improve performance. When I grouped 10 messages per query I was observing 10-20ms, again with a few very short and a few very long. So 10-20ms/10messages gives me an effective 1-2ms per message instead of the 5-15ms I had before.

My Objectives

• Determine how to model E, F, and G. I figured the easiest and most accurate method would be by experimentation, but I am not sure what data I should be looking to collect.

• Determine how many messages I should group together in order to minimize latency while maximizing the number of clients in stage A that I can have. I figured I could have a variable number of grouped messages based off of load, but I don't know how to come up with what this should look like.

• Finally be able to list a max number of clients I can have in stage A and say with some amount of certainty that their messages can be stored to the database in x time once it gets to stage C.

Any help would be great. Thanks!

Answer 1

Queue models depend on a few key distributions: the distribution of the time gaps between incoming jobs, the distribution of service times (how long it takes to process a job). Some commonly used models for these distributions are the exponential, gamma, and Weibull distributions. To find out which distribution is appropriate for your situation, you will need to collect data and do some model selection. You may also want to model the parameters for the distributions as time-varying, but this makes things much more complicated.

EDIT: "I found that each query took about 5-15ms with a few taking 1ms and a few taking 600+ms."

The Weibull distribution I mentioned before is often used to model "extreme" events.