Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm aware that there's been lots of advances with regards to image recognition, image classification, etc with deep, convolutional neural nets.
But if I train a net on, say, PNG images, will it only work for images so encoded? What other image properties affect this? (alpha channel, interlacing, resolution, etc?)
Short answer is NO.
The format in which the image is encoded has to do with its quality. Neural networks are essentially mathematical models that perform lots and lots of operations (matrix multiplications, element-wise additions and mapping functions). A neural network sees a Tensor as its input (i.e. a multi-dimensional array). It's shape usually is 4-D (number of images per batch, image height, image width, number of channels).
Different image formats (especially lossy ones) may produce different input arrays but strictly speaking neural nets see arrays in their input, and NOT images.
While Djib2011 answer is correct, I understand your question as more focused on how the image quality/properties affect neural network learning in general. There is only little research in this topic (afaik), but there might be more research on it in the future. I only found this article on it. The problem at the moment is, that this is more a problem appearing in practical applications and less in an academic research field. I remember one current podcast where researchers observed that even the camera that was used to take a picture could have a big effect.
This is a riff on the first answer from Djib2011. The short answer has to be no. Longer - Firstly photos are always encoded as a tensor as follows. An image is a number of pixels. If the photo is considered to have m rows and n columns, each pixel is specified by it's row and column location, that is by the pair (m,n). In particular there are m*n pixels which is very large even for 'small' photos. Each pixel of the photo is encoded by a number between zero and one (blackness intensity) if the photo is black and white. It is encoded by three numbers (RGB intensities) if the photo is color. So one winds up with a tensor that is either a 1xmxn or a 3xmxn. Image recognition is done through CNN's which, taking advantage of the fact that photos don't change that much from pixel to pixel, compress the data via filters and pooling. So the point is that CNN's work by compressing the incredibly large numbers of data points (or features) of a photo into a smaller number of values. So whatever format you start with, CNN's start off by further compressing the data of the photo. Hence the per se independence from the size of the representation of the photo.
However, a CNN will demand that all images being run through it are all of the same size. So there is that dependency that will change depending on how the image is saved. In addition, to the extent that different file formats of the same size produce different values for their tensors, one cannot use the same CNN model to identify photos stored by different methods.
While changes in camera or image compression after training can be severe, if it is the same, the problem is much less. Of course with more noisy images the performance is less, but I never heard that standard JPEG compression would make a big difference. But it will depend on the application.
If you change things after training, it very much depends. E.g. for some networks changing the resolution doesn't work at all. For others its possible. Its very network specific. In general any change (even lens, lighting, background, etc) needs to be evaluated and needs to be included in the training from a theoretical perspective.
In general its not a good idea to have training data that is qualitatively different. If you want to classify PNG and JPG, then it would be best to also train on both. The same for other image properties.
A CNN cannot extrapolate, it usually just works within the training set space. Other models can do that, like rule based models.
