Compare and quantify quality of data We get vendor quotes (their prices for different financial securities) every month. I want to compare which of these vendors 'agree' (i.e come close to) with our internal prices [the gold standard].
I also want to test these for sub categories - say Vendor A agrees with us on bonds, but vendor B agrees better with equities.
What statistical tests can I perform to rank the vendors (in terms of being close to our prices) and quantify the gap.
thanks.
 A: The simplest thing would be to look at the difference between your price and each vendor's price (in absolute value terms) and see which is highest each month. That gives you one rating for each vendor each month for each comparison you want to make (where comparisons would be "all", "equity", "bond" etc).
You could graph these differences over time to see if they change. 
Slightly more complex, you could set up a regression model for each month:
AVdiff ~ vendor + category
where AVdiff is the absolute value of the difference between prices and the other two are dummy variables. That would give you one model per month.
There are ways to try to account for multiple months at once (e.g. multi-level models, generalized estimating equations) but I don't think they are right here as your main interest seems to be in what would be the random effect (vendor) not in change over time. 
Time series models may be appropriate; I do not know enough about these to give a solid recommendation one way or the other, but usually they require quite long series and you may not have enough months. 
