How many examples do I need to show that a change in a feature causes a change in prediction? not being a statistician I have a bit of a trouble to wrap my head around the following concept.
Say I have a dataset $D$ of $N$ data points and a binary classifier (that was fit to some other dataset) that has mean accuracy $a$.
I want to change $D$ to $D'$ by altering the data points in some way and want to observe the new accuracy $a'$ of the classifier, expecting $a' < a$.
How do I choose the dataset size $N$ to be able to reliably say that the change was responsible for the performance drop? How do I report the confidence of that statement? Do I have to do a power analysis?
To instantiate a concrete example:
Say I have a trained neural classifier that classifies tweets in two categories (toxic/non-toxic). I create $D$ as a set of tweets where the accuracy of the classifier is high, say $a = 0.9$. I change $D$ to $D'$ in some defined way, say by inserting a certain word at the beginning of the tweet and expect the classifier performance to be significantly lower on $D'$, $a' < a$. How big do I need to choose $N = |D|$, i.e. how many tweets do I need to create, how do I compute the significance and how do I report the confidence?
Thank you all very much!
 A: great question. I know you're asking specifically about neural networks and about sample size. I will try to answer a bit indirectly by going over a few similar techniques that I'm familiar with. 
In random forests there are a series of feature selection techniques that do exactly this type of analysis where they test the strength of a predictor by seeing how the accuracy changes after permuting the value. They don't really take sample size into account explicitly, but this might be another approach to pursue. They usually use some variant of the algorithm below, and then either create distributions for changes in errors or use user-defined cutoffs to identify important features. Some of these algorithms are hypothesis-based and some are not, so it gives you an idea of what sorts of tools are available. 
The general algorithm goes as follows:
General Permutation Algorithm


*

*Train a random forest, report scores & rank features

*Shuffle or permute the values of the features in $D$

*Train a new RF with the permuted features

*Compare the feature importance before and after, and keep features exceeding a threshold


References to Permutation Techniques in RFs
Below are a few papers that deal with permuting features as a strategy to identify which have the strongest relationships to your target. 


*

*Altmann, A., Toloşi, L., Sander, O., & Lengauer, T. (2010). Permutation importance: a corrected feature importance measure. Bioinformatics, 26(10), 1340-1347.

*Strobl, C., Boulesteix, A. L., Kneib, T., Augustin, T., & Zeileis, A. (2008). Conditional variable importance for random forests. BMC bioinformatics, 9(1), 307.

*Janitza, S., Strobl, C. and Boulesteix, A.L., 2013. An AUC-based permutation variable importance measure for random forests. BMC bioinformatics, 14(1), p.119.
Hope this helps. 
