Skip to main content
Cross Validated

Return to Question

Edited title and question for clarity
Source Link
dev01
dev01
  • 1
  • 1

How many test executions are required (and how many must pass) to say my software works 90% of the time with 95% confidence?

I have some software that operates non-deterministically. Running a test on the code can result in a pass or fail depending on unknown external factors. Assuming that the odds of passing or failing are equal because of these factors, how many times must I run the test, and how many times must the software pass the test so that I can say with 95% confidence that the software works as expected 90% of the time?

Edit: Assume that the odds that an external factor causes the test to fail are unknown instead of a pass/fail being equal.

How many test executions are required (and how many must pass) to say my software works with 95% confidence?

I have some software that operates non-deterministically. Running a test on the code can result in a pass or fail depending on unknown external factors. Assuming that the odds of passing or failing are equal because of these factors, how many times must I run the test, and how many times must the software pass the test so that I can say with 95% confidence that the software works as expected?

How many test executions are required (and how many must pass) to say my software works 90% of the time with 95% confidence?

I have some software that operates non-deterministically. Running a test on the code can result in a pass or fail depending on unknown external factors. Assuming that the odds of passing or failing are equal because of these factors, how many times must I run the test, and how many times must the software pass the test so that I can say with 95% confidence that the software works as expected 90% of the time?

Edit: Assume that the odds that an external factor causes the test to fail are unknown instead of a pass/fail being equal.

Source Link
dev01
dev01
  • 1
  • 1

How many test executions are required (and how many must pass) to say my software works with 95% confidence?

I have some software that operates non-deterministically. Running a test on the code can result in a pass or fail depending on unknown external factors. Assuming that the odds of passing or failing are equal because of these factors, how many times must I run the test, and how many times must the software pass the test so that I can say with 95% confidence that the software works as expected?