Return to Answer

6. Look at vision algorithms for color constancy. The computer vision community has studied color constancy, the problem of correcting for an unknown illumination source; see, e.g., this overview. You might explore some of the algorithms derived for this problem; they attempt to infer the illumination source and then correct for it, to derive the image you would have obtained had the picture been taken with the reference illumination source.

6. Look at vision algorithms for color constancy. The computer vision community has studied color constancy, the problem of correcting for an unknown illumination source; see, e.g., this overview. You might explore some of the algorithms derived for this problem; they attempt to infer the illumination source and then correct for it, to derive the image you would have obtained had the picture been taken with the reference illumination source.

7. Look into Color Constant Color Indexing. The basic idea of CCCI, as I understand it, is to first cancel out the unknown illumination source by replacing each pixel's R value with the ratio between its R-value and one of its neighbor's R-values; and similarly for the G and B planes. The idea is that (hopefully) these ratios should now be mostly independent of the illumination source. Then, once you have these ratios, you compute a histogram of the ratios present in the image, and use this as a signature of the image. Now, if you want to compare the image of the coin $C$ to a reference image $R$, you can compare their signatures to see if they seem to match. In your case, you may also need to adjust for angle if the picture of the coin $C$ was not taken head-on -- but this seems like it might help reduce the dependence upon illumination source.

6. Look at vision algorithms for color constancy. The computer vision community has studied color constancy, the problem of correcting for an unknown illumination source; see, e.g., this overview. You might explore some of the algorithms derived for this problem; they attempt to infer the illumination source and then correct for it, to derive the image you would have obtained had the picture been taken with the reference illumination source.

I don't know what might work, but in the spirit of brainstorming, I'll share some ideas you could try:

1. It looks like Hue gave you a pretty good discriminator between silver and copper/gold, though not between copper and gold, at least in the single example you showed here. Have you examined using the Hue in greater detail, to see whether it might be a viable feature to distinguish silver from copper/gold?

2. Similarly, you might try transforming to rg chromacity and then experimenting to see whether the result is helpful at distinguishing silver from copper/gold. It is possible that this might help adjust for illumination variation, so it could be worth trying.

3. I gather that, from the ratios of coin sizes (radiuses), you have an initial hypothesis for the type of each coin. If you have $n$ coins, this is a $n$-vector. I suggest you test this entire composite hypothesis in a single go, rather than $n$ times testing your hypothesis for each coin on its own.

Why might this help? Well, it may let you take advantage of the relative hues of the coins to each other, which should be closer to invariant with respect to illumination (assuming relatively uniform illumination) than each coin's individual hue. For example, for each pair of coins, you can compute the difference of their hues and check whether this corresponds to what you'd expect give your hypothesis about their two identities. Or, you could generate a $n$-vector $p$ with the predicted hues for the $n$ coins; compute a $n$-vector $o$ with the observed hues for the $n$ coins; cluster each one; and check that there is a one-to-one correspondence between hues. Or, given the vectors $p,o$, you could test whether there exists a simple transformation $T$ such that $o \approx T(p)$, i.e., $o_i \aapprox T(p_i)$ holds for each i. You may have to experiment with different possibilities for the class of $T$'s that you allow. One example class is the set of functions $T(x)=x+c \pmod{360}$, where the constant $c$ ranges over all possibilities.

4. Rather than using the color of the coin, you might consider trying to match what is printed on the coin. For instance, let's say that you have detected a coin $C$ in the image, and you hypothesize it is a one pound coin. You could take a reference image $R$ of a one pound coin and test whether $R$ seems to match $C$.

If images of the coins aren't taken from directly head-on, then as a first step you may want to compute the ellipse that represents the perimeter of the coin $C$ in the image and infer the angle at which the coin is being viewed. This will let you compute what $R$ would look like at that angle, before performing the matching.

In the spirit of brainstorming, I'll share some ideas you could try:

1. Try Hue more? It looks like Hue gave you a pretty good discriminator between silver and copper/gold, though not between copper and gold, at least in the single example you showed here. Have you examined using the Hue in greater detail, to see whether it might be a viable feature to distinguish silver from copper/gold?

2. Other color spaces? Similarly, you might try transforming to rg chromacity and then experimenting to see whether the result is helpful at distinguishing silver from copper/gold. It is possible that this might help adjust for illumination variation, so it could be worth trying.

3. Check relative differences between coins, rather than looking at each coin in isolation? I gather that, from the ratios of coin sizes (radiuses), you have an initial hypothesis for the type of each coin. If you have $n$ coins, this is a $n$-vector. I suggest you test this entire composite hypothesis in a single go, rather than $n$ times testing your hypothesis for each coin on its own.

Why might this help? Well, it may let you take advantage of the relative hues of the coins to each other, which should be closer to invariant with respect to illumination (assuming relatively uniform illumination) than each coin's individual hue. For example, for each pair of coins, you can compute the difference of their hues and check whether this corresponds to what you'd expect give your hypothesis about their two identities. Or, you could generate a $n$-vector $p$ with the predicted hues for the $n$ coins; compute a $n$-vector $o$ with the observed hues for the $n$ coins; cluster each one; and check that there is a one-to-one correspondence between hues. Or, given the vectors $p,o$, you could test whether there exists a simple transformation $T$ such that $o \approx T(p)$, i.e., $o_i \approx T(p_i)$ holds for each i. You may have to experiment with different possibilities for the class of $T$'s that you allow. One example class is the set of functions $T(x)=x+c \pmod{360}$, where the constant $c$ ranges over all possibilities.

4. Compare to reference images? Rather than using the color of the coin, you might consider trying to match what is printed on the coin. For instance, let's say that you have detected a coin $C$ in the image, and you hypothesize it is a one pound coin. You could take a reference image $R$ of a one pound coin and test whether $R$ seems to match $C$.

If images of the coins aren't taken from directly head-on, then as a first step you may want to compute the ellipse that represents the perimeter of the coin $C$ in the image and infer the angle at which the coin is being viewed. This will let you compute what $R$ would look like at that angle, before performing the matching.

5. Check how color varies as a function of distance from the center? Here is a possible intermediate step in between "the coin's mean color" (a single number, i.e., 0-dimensional) and "the entire image of the coin" (a 2-dimensional image). For each coin, you could compute a 1-dimensional vector or function $f$, where $f(r)$ represents the mean color of the pixels at distance approximately $r$ from the center of the coin. You could then try to match the vector $f_C$ for a coin $C$ in your image against the vector $f_R$ for a reference image $R$ of that coin.

This might let you correct for illumination differences. For instance, you might be able to work in grayscale, or in just a single bitplane (e.g., Hue, or Saturation, or Brightness). Or, you might be able to first normalize the function $f$ by subtracting the mean: $g(r) = f(r)-\mu$, where $\mu$ is the mean color of the coin -- then try to match $g_C$ to $g_R$.

The nice thing about this approach is that you don't need to infer how much the coin was rotated: the function $f$ is rotation-invariant.

If you want to experiment with this idea, I would compute the function $f_C$ for a variety of different example images and graph them. Then you should be able to visually inspect them to see if the function seems to have a relatively consistent shape, regardless of illumination. You might need to try this for multiple different possibilities (grayscale, each of the HSB bitplanes, etc.).

If the coin $C$ might not have been photographed from directly head-on, but possibly from an angle, you'll first need to trace the ellipse of $C$'s perimeter to deduce the angle from which it was photographed and then correct for that in the calculation of $f$.

I don't know if any of these has a chance of working, but they are some ideas you could try.