I am currently trying to design an experiment that will be testing how fast a conveyor is able to convey different size cases. The case sizes are based on 4 dimensions (LxWxH and Weight) and the case population size is well over 100,000 combinations.

I am looking for some insight on what kind of sample size I need to test to be able to be 95% confident that I have accounted for the 100,000 possible combinations. What kind of statistical analysis can I run on the population data that can tell me the how many and what size cases I need to test in my experiment that will represent the overall case population?

Just looking for some direction so that I can do further research on my own.

EDIT: I apologise for the delay in answering your questions.Experimentation is not expensive but is time constrained. I have 120 days to experiment and can most likely achieve 1 experiment run per day and therefore get 120 data points. The data is continuous and normal around a central mean I heave measurements on all four factors I agree that a full factorial design is applicable and is the approach I would like to take. Kjetil, I believe you are saying that I can use the population data I have to break the factors into a HIGH/LOW category by looking at their distribution and choosing percentage values accordingly?

A little more background on the experiment is that we are unloading these cases from a trailer and when we experiment we are only able to measure an unload rate (Cases / Hour) for the entire trailer and not by each type of case. My thought is that I can take the case data that I have and break the cases into groups based on the four factors (example - Small, Medium, Large) and those would be my experiment factors. My levels would then be the percentage % of the each of those categories that we unload from the trailer (i.e we unload a trailer that is 25% Small, 50% Medium, 25% Large).

  • 1
    $\begingroup$ So, you have four factors, so you could run a $2^4$ factorial design, as an initial experiment. From that you can learn about linear effects and interactions, and about the variance. Add some central points to learn about possible curvature. From such an initial experiment you can hopefully learn enough to answer your question. From the model estimated from your initial pilot experiment, make perdictions for variable values/combinations not used in the pilot, to test your model. How much do an individual experimental run cost, and what is your budget? With a little more info we could .... $\endgroup$ – kjetil b halvorsen Feb 17 '16 at 15:16
  • $\begingroup$ ... more. If you know there are curvatures, you could start with an $3^4$ experiment instead of my proposed $2^4$ factorial. $\endgroup$ – kjetil b halvorsen Feb 17 '16 at 15:17
  • $\begingroup$ If you want a better answer you really need to try to answer some af the questions in my comments. $\endgroup$ – kjetil b halvorsen Feb 18 '16 at 15:45
  • $\begingroup$ Do you have any knowledge as to the distribution of the suitcases? Are they discrete uniform in the $100.000$ possible shapes? Roughly normal around a certain mean point? It is not possible to answer your question without knowing/guessing/assuming a distribution of the suitcases. $\endgroup$ – Jeremias K Feb 18 '16 at 20:14

As said in comments, we really need more information for a good/full answer. But here is a few points to get you started. You will probably need to experiment in stages, so this ideas are for the first stage.

You have four variables. Are experimentation expensive? So start with a full $2^4$ factorial design, or if that is too expensive, you can use a half fraction. Add in a few centerpoints to get information on possible nonlinearities. If possible add in a few more replicates (on some of the other points) to get a better idea about variance. Randomize the order of the runs! Maybe it is good to block the experiments in time.

From the analysis of this first stage, you will find out if nonlinearities or interactions are needed. Or, maybe, if the variance is large, you can only conclude that you need to replicate the first stage to get good enough estimates.

Then build a predictive model, and test it by doing predictions for factor values not used in the experiment. Then you will find if the model is good enough.

At this point it is impossible to answer your question about the necessary sample size.

And, if you need a reference for the concepts used here, as factorial designs, a half factorial, blocking, this book is the best there is: http://www.amazon.com/Statistics-Experimenters-Innovation-Discovery-Edition/dp/0471718130

EDIT (after OP comment below)

Testing criteria range? You have four variables measured on the objects (cases): Length, Width, Height, Weight. Do you have measurements of all four on the 100000 objects? Are they heavily correlated or not? Correlation might be a problem for the factorial design I have proposed, because then the design might use combinations that are rather atypical for the population of objects. For a moment, forget about this. Make plots of the four variables, and choose low and high values for experimentation as some percentage points from that distribution. If the correlation (discussed above) is high, choose percentage points far from the extremes (maybe 20%, 80% or 25%, 75%) so that the resulting factorial cube does not contain impossible or very atypical combinations. If that do not work out, maybe leave the factorial idea and go for a D-optimal design (use CRAN package AlgDesign) but that should be a last resort, as factorial designs are easier to interpret. Hope this helps. Note that D-optimal designs tend to choose extremal design points, so might be bad if non-linearities are important or the model gets worse near the design boundary. So use with care!

| cite | improve this answer | |
  • $\begingroup$ I added a bounty on this question because I was especially interested in the "testing criteria range" part of the question. I get that sample size is hard to discuss here, but I'll be happy to award the bounty if you think you can talk a bit more about choosing experimental test cases when fully enumerating is impossible. Unless I misunderstood your answer, I don't see that addressed here $\endgroup$ – shadowtalker Feb 23 '16 at 11:57
  • $\begingroup$ Will se what I can do ... $\endgroup$ – kjetil b halvorsen Feb 23 '16 at 12:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.