How many data points for my new product do I need to tell if my new product performs differently to the old? We've been using the old formula to produce a product for years. We take a sample every quarter or so and run it through 8 different quality tests (every quarter because it takes that long for the tests to complete). Currently 53 samples of the old formulation have been put through these tests.
One of the components of the old formula became unavailable so we're using a new formula that replaces the unavailable component with what we believe is a suitable substitute. Again, every quarter or so, we're going to take a sample and run it through the same tests as we did for the old formula.
How many samples of the new formula are we going to have to test to be confident we can tell whether the new formula is "significantly different" or "not significantly different" to the old formula? 
(Whether it is significantly different or not is beside the point, I just want to know when we can be 95% confident that we are reaching the right conclusion - different or not different - rather than having to wait for 53 samples and 13 years to find out).
If the answer to this question is "it depends", could someone please explain what it depends on? Much gratitude.
 A: This is more of an extended comment than an answer, sorry.  The concept you are after is "power", and the text below might help you get started.
It's useful in these situations to be very careful with terminology.  I advocate saving the word "significant" to describe statistically significant evidence; and using "material" to mean significant in the way you use it.  Otherwise one quickly becomes tangled up on "statistically significant".  Statistically significant actually is usually shorthand for "there is evidence of a non-zero difference between the two".  If you get a big enpough sample almost inevitably you will reach this point.
One of the things "it depends" on then is how material a difference has to be before it matters to you.  Ie is a 5%, 1%, 0.1% deviation in quality the maximum permissable.
Another thing is whether you are interested only in detecting the bad case (new formula is materially worse) or in either direction of change (materially different).  
A third thing that makes a big difference is the variance in measurements of the quality.  Is each of your 53 observations so far identical (probably not), and if not, how much do they vary (ie what is their standard deviation)?  A possible complication is in if the quality has been changing systematically over time eg improving as techniques got better; so you would need to carefully look at that too.
The final thing it depends on is how much different the formulae in reality are - which obviously you don't know.  Typically we can get around this by calculating the power of a statistical test as a function of the unobserved population difference ie we can say "if the real difference is Y, the probability of us incorrectly concluding from our observations so far that the difference is zero is X", and calculating this for all plausible Ys (you will see from this why it is important to think through what difference would be material for you).
