22
$\begingroup$

I'm aware that there's been lots of advances with regards to image recognition, image classification, etc with deep, convolutional neural nets.

But if I train a net on, say, PNG images, will it only work for images so encoded? What other image properties affect this? (alpha channel, interlacing, resolution, etc?)

$\endgroup$
  • 2
    $\begingroup$ I don't have enough experience with the statistics of images to give a confident answer, but I would expect that different models use completely different schemes to code images into features, some of which will be affected by lossy JPEG compression, alpha channels, etc., and some of which won't, with the details being model-specific. I know that there exist both models that treat each pixel as a feature and models that carve images into regions and use region properties as features. $\endgroup$ – Kodiologist Jun 18 '17 at 0:27
20
$\begingroup$

Short answer is NO.

The format in which the image is encoded has to do with its quality. Neural networks are essentially mathematical models that perform lots and lots of operations (matrix multiplications, element-wise additions and mapping functions). A neural network sees a Tensor as its input (i.e. a multi-dimensional array). It's shape usually is 4-D (number of images per batch, image height, image width, number of channels).

Different image formats (especially lossy ones) may produce different input arrays but strictly speaking neural nets see arrays in their input, and NOT images.

$\endgroup$
4
$\begingroup$

While Djib2011 answer is correct, I understand your question as more focused on how the image quality/properties affect neural network learning in general. There is only little research in this topic (afaik), but there might be more research on it in the future. I only found this article on it. The problem at the moment is, that this is more a problem appearing in practical applications and less in an academic research field. I remember one current podcast where researchers observed that even the camera that was used to take a picture could have a big effect.

$\endgroup$
  • $\begingroup$ Could you point me to the podcast please? $\endgroup$ – David Ernst Sep 27 '17 at 16:13
  • $\begingroup$ It's a one hour episode of "talking machines", where they have a discussion with a researcher practicing machine learning in Africa (ca. middle to end) iTunes link: itunes.apple.com/de/podcast/talking-machines/… $\endgroup$ – Bobipuegi Sep 27 '17 at 16:51
2
$\begingroup$

This is a riff on the first answer from Djib2011. The short answer has to be no. Longer - Firstly photos are always encoded as a tensor as follows. An image is a number of pixels. If the photo is considered to have m rows and n columns, each pixel is specified by it's row and column location, that is by the pair (m,n). In particular there are m*n pixels which is very large even for 'small' photos. Each pixel of the photo is encoded by a number between zero and one (blackness intensity) if the photo is black and white. It is encoded by three numbers (RGB intensities) if the photo is color. So one winds up with a tensor that is either a 1xmxn or a 3xmxn. Image recognition is done through CNN's which, taking advantage of the fact that photos don't change that much from pixel to pixel, compress the data via filters and pooling. So the point is that CNN's work by compressing the incredibly large numbers of data points (or features) of a photo into a smaller number of values. So whatever format you start with, CNN's start off by further compressing the data of the photo. Hence the per se independence from the size of the representation of the photo.
However, a CNN will demand that all images being run through it are all of the same size. So there is that dependency that will change depending on how the image is saved. In addition, to the extent that different file formats of the same size produce different values for their tensors, one cannot use the same CNN model to identify photos stored by different methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.