Should image classifier be trained using colormap pixels or the actual value? For example, I have a population density map of a 100 x 100 km square region. 
Each part of the rectangular region represents the population density i.e. (1,1) -> 128 people, (100,100) -> 50 people
I plug this square matrix into MATLAB using the "imagesc" command and it generates an image presenting the population density.

Now the question is that I wish to train a set of these 100 x 100 images using an ANN classifier.
Should I use the pixel value of the colormap generated by imagesc (RBG colors), or should I use the values of the population density matrix (number people per region)? 

 A: Use the raw data! 
If I understand your problem correctly, you've got a matrix of data points where the (x,y) coordinates indicate a location and the value is the population density at that location. In other words $\textrm{data}(50,75)=100$ indicates that the area at position (50,75) has a population density of 100 people/square mile.
You have also generated a raster plot of this data with imagesc.
If this is true, there are several reasons to use the raw data


*

*The raw data is univariate, but the colormap pixels are not. By that, I mean that the (x,y) coordinates are associated with a single number (people/square mile). On the other hand, if you use the colormapped values, you'll now have three outputs (the r,g and b values) for each (x,y) coordinate. Predicting multiple outputs is typically much harder.

*Worse, the color map is totally arbitrary. Your response variable presumably changes smoothly with population density--the areas with 10 people/square mile should be relatively similar to those with 5 and 15. This ordering, which you get for free when using the numeric values, can be distorted by applying a color map: is (0,255,0) more like (0, 0, 255) or (255, 0, 0)? This transformation also varies depending on which color map you choose.

*Finally, converting to a color map may discard information. In theory, the colormap space is pretty big--there are $256^3$ possible colors. If your data points are distributed evenly across their range, you could potentially end up one color/entry. However outliers can eat into that dynamic range and you may find that a lot of the data is "compressed" into the middle of that range (the bluish green in your example).
In contrast, I can't think of a single good reason to use the colormap values instead. Perhaps it would make sense if you expect to receive future data that only comes in this form (e.g., scans from older documents), but otherwise...it just seems odd. It makes your classification problem harder in several different ways for no real reason. 
A: Using RGB images is to expensive. 100x100 matrix of data means that you must design network which contains 10000 inputs. For RGB format of data must be three times more inputs, because you use 3 parameters for every point in matrix, so you need 3x100x100 or 30000 inputs and in first layer your computations will be processed longer if you will use RGB images as input parameters.
