Suppose I train a neural network on dataset A and evaluate on dataset B (that has a different feature distribution than dataset A). If I increase the amount of data in dataset A by a factor of 10, is it likely to decrease accuracy on dataset B?
On the contrary, more data is almost always better at generalizing to unseen data. The more examples of the data-generating process, the closer the model predictions will get to that of the population. After all, your model has seen a larger part of the population.
Hypothetically, if all hyperparameters were to be held constant, then more data means more steps along the gradient at the same learning rate, which could indeed overfit more easily. However, if you regularize appropriately, choose the right learning rate, etc., then this isn't a problem.
That said, if the new and old data do not come from the same distribution, then simply adding more data will not remedy this. You should probably look into over-/undersampling, or other methods, depending on what exactly you mean by a different feature distribution.
The fact that dataset B "has a different feature distribution than dataset A" makes the question fairly ambiguous. It's equivalent to asking how well will a neural net trained for problem A work on problem B - there is no definitive answer. But yes, it is possible that training on more samples from dataset A will make your neural net perform worse on dataset B.
One example where this may come up is if you're training your neural net on simulated data and validating on "real world" data. Because the simulated data doesn't perfectly represent the real data, the neural net may learn patterns in the simulated data that don't generalize to the real world. In that case there is likely to be a training set size that optimizes your performance on the validation set and additional training points will reduce the validation accuracy. This is not a great way to go about things though.
This issue is not what overfitting typically refers to, but it does have an analogous nature (perhaps someone else can help with an exact term for this).
Yes it can. One possibility is that all the examples in dataset A are similar, so your classifier may be overfitting this dataset, and may work worse on dataset B. It might well be your case, if you are fitting the features very common in A but nearly absent in B.
Another possibility is that you are overfitting the same features of A and B. Adding more samples to A will make this dataset less biased, but the classifier will work worse on B.
This is not specific to neural networks.
Some theoretical considerations.
The book "Probabilistic Learning" states that "Neural networks with one hidden layer are universally consistent if the parameters are well-chosen." They mean that, as training set size goes to infinity, the error rate converges to error of the Bayes classifier.
The book "Understanding machine learning" gives an estimate of VC-dimension of the class of hypotheses of NN. Using Fundamental Theorem of Learning may give an idea, how big shall be a training set to get the accuracy you want. Usually, it is huge.
Neither of these results mean that if you increase training set 10 times, the accuracy will be better. It only mean that if you increase your training set indefinitely, then, eventually, the results will get better. But then, they do not say how to select the "well - chosen" parameters. So, yes, increasing size of data 10 times may get worse results from theoretical point of view.
The data in your training and test sets can be modeled as h(x) + noise. In this context, the noise is the variability in your training and test data that is not explained by some common (theoretically optimal) model h(x). The important thing here is that, for example, if your training and test sets are sampled from entirely different distributions, then ALL of your data is noise, even if on their own, both training and test set data are very well structured. In this case, even a model with 1 or 2 parameters will be overfitting right away - regardless of how many data points you have in your training set!
In other words - the greater the amount of noise in your data, the easier it is to overfit and the simpler model you are restricted to using. With, say, gaussian noise, increasing the amount of data in your training set increases the data-to-noise ratio, reducing overfitting. If your training and test data are from (slightly) different distributions, increasing the amount of data will do nothing to reduce this source of noise! The data-to-noise ratio will stay the same. Only other sources of noise will be eliminated (e.g. measurement noise, if applicable).
So increasing the amount of data can only make overfitting worse if you mistakenly also increase the complexity of your model. Otherwise, the performance on the test set should improve or remain the same, but not get significantly worse.