How much resources does preprocessing generally take? I'm learning about CNNs (Convolutional Neural Networks) and the wiki page says, 

Convolutional networks [...] are variations of multilayer perceptrons which are designed to use minimal amounts of preprocessing."

So could one say that a characteristic of the variations of multilayer perceptrons leads to minimal amounts of preprocessing?
If so, how much resources does preprocessing usually take up?
Does using this technique of data mining introduce a new model for minimizing preprocessing?
 A: The problem with preprocessing isn't (necessarily) that it's computationally expensive. It's that it's hard to design good features. When we have a lot of data, and with careful design of the model, we can sometimes have models such as CNNs that can find better features than we can come up with manually.
The primary application of CNNs is in image processing. One of the conceptually simpler tasks in that space is called scene recognition: given an image, what kind of place is it a picture of? A city, the coast, a forest, ...?
Before about five years ago, the basic standard method for doing this went something like:


*

*Extract a dense grid of feature vectors each image in your training set: maybe SIFT, HOG, SURF, or any of many other hand-designed options. You also have to choose a reasonable grid size and stride, whether you want multi-scale grids, and so on.

*Run k-means clustering on a sample of those feature vectors. Make sure to choose the right $k$ here: too high or too low will both give you bad results.

*Quantize all of the features in the dataset according to this clustering, and represent each image by the histogram of which centers the vectors fall into (the bag of words model).

*Take the square root of each entry in the histogram, because it seems to help. (This corresponds to using the Hellinger distance between them.)

*Train a linear SVM or other classifier on these vectors.


All of the steps prior to step 5 have to be pretty manually developed, especially step 1, and there's no guarantee that what works well for one kind of dataset will work well for others too. If you pick bad features in step 1, your learning algorithm has no chance of recovering from that.
Now, the standard technique is:


*

*Resize your images to $256 \times 256$. (Obviously there's nothing magical about this size.)

*Train a CNN with stochastic gradient descent.


There's still a lot of human work that goes into picking the CNN architecture. But that's often more transferrable between different kinds of images, and because we're optimizing the whole model at once, the feature extractors (the early parts of the network) can co-adapt with the classifier (the end of the network) to find features that work well for this dataset. Doing so typically only works when you have a lot (millions) of images, though; there are just too many parameters in the network to deal with fewer.
