Singular value decomposition on RGB images Is singular value decomposition (SVD) only done for grayscale images? All examples in the literature seem to focus on grayscale images. 
I was wondering if SVD also makes sense if applied to each of the channels of an RGB image.
 A: Nope; SVD decomposition is not only for gray scale images. Having said that it does not make much sense to do it in lossy formats. (Specialised applications like forgery detection, watermarking etc. are exceptions to this statement.)
Usually colour images are compressed in some way. As you correctly notice that means that in the case of an RGB image you will have three matrices. The fact is though that these numbers are represented in a lossy format that uses integers.
During the RGB conversion you have already squeezed out redundant information that SVD would allocate to lower variance components. This translates in poor gains in terms of compression. Furthermore the fault tolerance will be very low exactly because the whole idea of a compression algorithm is that a few bits pack a lot of information. Clearly if you get only a few bits off you might end up way off in terms of reconstruction. This translates in poor reconstruction quality.
Finally SVD assumes that your data are unbounded and are not represented exclusively as integers. This means that for your final reconstructions you will have to truncate and/or round-off your estimates. This is not a total catastrophe (PCA works reasonably good with discrete values after all) but it will introduce some sort of arbitrary truncations that given the points mentioned above, can only make things worse.
So the reason people use mostly grey scale images is simple: RGB images would make poor test-beds for generic SVD applications. 
You can certainly do SVD in each of the three matrices RGB images though. The SO link here: "Colored Pixels Appearing When Trying to Compress Image (pics included)" presents a more detailed discussion on the matter if you are interested.
