Here's what my data look like:
From a mark-recapture study on a population of birds, I have data on the coloration score for each individual bird. The coloration score is a simple visual score that places an individual into one of four color categories: Yellow, Patchy Orange, Uniform Orange, or Red. Coloration was scored for each individual every time an individual was captured (note: different individuals were not captured the same number of times). Only one observer (me) assigned coloration scores throughout the study. The study took place over a short enough time scale that we can reasonably assume the coloration of a given individual does not change over time (at least not enough to 'move' a given individual from one color category to another)
I want to evaluate how repeatable my assessment of coloration was during the study. As I understand it, my data seem to violate many of the features of data that can be analyzed with common measures of repeatability.
Can anyone provide me with any suggestions for measurements of repeatability that I might use?