How much data is needed to train CNN from scratch? Any rule of thumb, on how many input images would be needed to have a reasonable chance not to overfit the data when training a CNN from scratch?
In other words, what is a reasonable amount of data (in order of magnitude) needed to train a CNN from scratch?
For instance, for ResNet 50 that has about 26M parameters.
 A: There's no real rule of thumb to this, as it highly depends on the classification/regression problem and the nature of your images. Generally speaking, you need thousands, but usually, orders of magnitude more. There are smaller examples, e.g. the LUNA16 lung nodule detection challenge only has around 1000 images.. When your data is small, you hopefully have the ability to either heavily regularize the model, or provide plenty of initial image transformations (shifts, scalings, etc.). 
A nice reference is "Understanding Deep Learning Requires Rethinking Generalization", where the authors show that a typical CNN can easily memorize all of ImageNet (A million images) even when the labels are totally random, i.e. training error goes to 0. So 30 million parameters is plenty for memorizing huge amounts of data.
With that being said, you're much more likely to get better results by not starting from scratch, instead opting to use a pretrained CNN (on say, ImageNet), as the bulk of the filters it learns tend to generalize quite well to other tasks, and will greatly reduce overfitting in the earlier stages of your training. 
To be clear, there is almost no reason to start from complete scratch. There's a great example of this from the 20bn-kitchenware dataset, where the task is to predict whether actions are successful (e.g. did the person pick up the cup successfully, or fail?). This paper: "The more fine-grained, the better for transfer learning" - Mahdisoltani, et. al. has a nice summary in Figure 4 of baselines tried for this, including pertaining on ImageNet and pretaining on predicting whether videos are running forwards or backwards. The forwards-backwards pertaining was a brilliant inspiration, because it not only teaches the CNN to recognize cause-and-effect, but also inadvertently teaches the CNN about how gravity works! 
