This is much more of a computer-related question but I believe this requires statistical knowledge (which I'm not well versed and currently reading on the basics) so i hope you can bear with me.

I have below as factors:

G = number of groups
n(1), n(2), .., n(G) = number of records in G (n varies in each group)

There is also a constant T which is number of threads where the processes runs concurrently. In this case, T = 4.

For example, G = 5 and below are the n values for each group:
1 - 10000
2 - 8581
3 - 1500
4 - 15678
5 - 7546

Since T = 4, only the first 4 will be accommodated and processed concurrently. Group 5 will be waiting until one of the process finishes.

Assuming they finished with below elapsed times... Thread --> n records --> elapsed time T1 --> 10000 records --> 25 s
T2 --> 8581 records --> 23 s
T3 --> 1500 records --> 3 s
T4 --> 15678 records --> 28 s

Since T3 took only 3 seconds to process, it will be the first to finish. Group 5 will then be accommodated in T3.
T3 --> 7546 records --> 20 s

Hence, the total runtime for all 5 groups will be 28 seconds because it is the last process to finish. Group 5 will finish at the same time Group 2 finishes (23 s). Elapsed Time = 28 s

Given the above example, I am having difficulty on how to derive an estimate given the number of groups and the number of records per group because G and n can vary (e.g. input can be any number of groups with varying records per group). If this is to be solved using a statistical approach, what method should I use that best fits this case? I have read on the fundamentals about linear regression but there are other regression types as well. I also have historical data on previous runs as well.

Any ideas / insight is very much appreciated. Thank you in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.