How many user interactions do I need to perform search relevance experiment?

I know similar questions have been asked many times here. But I still don't get simple formula or idea how estimate this.

I have site with traffic and I want to split portion of it to test new search relevance algorithms, i.e., result sorting. I will measure abandonment rate (searches without click/total amount of searches). I want to get this statistics with $\pm2\%$ accuracy with $95\%$ confidence.

Is there simple heuristic (something practical) I can apply to get required number of experiments ($10^3,~10^6,~\ldots$)?