You need to determine a margin of error. The margin of error tells you, roughly, the likely amount of sampling error from a sample of observations.
Since your outcome is binary the margin of error is, roughly $^1$,
$$ MOE \approx \dfrac{1}{\sqrt{N}} $$
So if you pick a margin of error, you can solve for $N$, leading to
$$ N \approx \dfrac{1}{MOE^2} $$
Here is a hypothetical example. Say I want to know the success rate of the machine to within 1% (equivalently, 0.01). My margin of error is then 0.01, and using this formula I would need 10, 000 samples since
$$ N = \dfrac{1}{0.01^2} = 10,000 $$
I could then claim I have estimated the success rate of the machine (assuming I did a simple random sample, etc etc) to within a margin of error of 0.01, meaning -- again, roughly -- the the true success rate could be my estimate plus or minus 1%.
We can not tell you what your margin of error should be. This depends on your constraints and context. That being said, this should tell you how many samples you need as a function of how certain you need your estimate to be.
Footnotes
- The actual margin of error for a binomial random variable is
$$ z_{1-\alpha/2} \times \sqrt{\dfrac{p(1-p)}{N}} $$
and is equal to the radius of the Wald confidence interval for the binomial parameter. Using the typical 95% confidence interval, $z_{1-\alpha/2} \approx 2$. Since $p$ is unknown, and the margin of error is maximized when $p=0.5$, we can actually create an upper bound for the margin of error
$$ MOE = z_{1-\alpha/2} \times \sqrt{\dfrac{p(1-p)}{N}} \leq 2 \sqrt{\dfrac{0.5^2}{N}} \>.$$
Hence, $N \leq \dfrac{1}{MOE^2}$ is likely an over estimation of the number fof required samples, but will guarantee a margin of error of at most $MOE$.
