How do I compare a matrix of population values to a matrix of sample values?

Let's say I have a matrix where the underlying data is from the entire population. Let's say I also have a matrix where the underlying data has been sampled from population.

How would I analyze how well the the sample represents the population?

My gut instinct is to convert it into a row-col listing:

row col popvalue samplevalue
1   1   34.5     33.2
[...]
i   j   54.4     51.2


and then run ANOVA between the two. Does this sound right or is there a better way to measure this?

• This question might answer itself once you elucidate what it means to you, in this application, for a sample to "represent" a population. For instance, in some applications you want to estimate the population mean from the sample; in others you want to estimate an extreme (such the chance of a flood or nuclear disaster); in others you want the sample to reproduce yet other aspects of the population. In this instance, perhaps (hypothetically) you want the eigenvalues of the sample matrix to reflect those of the population matrix. If so, that indicates a completely different analysis. – whuber Mar 29 '11 at 22:49
• Could you be more specific about what you mean by 'data is from the entire population'? ANOVA is a method than can be used on samples from large populations. When you have the data of an entire population the methods to use are different. – GaBorgulya Mar 30 '11 at 0:45

2 Answers

This is difficult to answer because the within-cell variance is unknown. The variances of the popvalue and samplevalue will give you only between-cell variance.

With two-way ANOVA, you could model the average difference between popvalue and samplevalue by row and by column. This doesn't tell you that much. As @whuber mentioned, you may be interested in a different parameter. Moreover, you may be interested at cell-level variation rather than just row- or column- level variation.

Another thing to consider: If you convert it to that sort of row-column listing and treat each cell as independent, you'll lose information about spatial correlations.

If these are aggregated data, I think you'll want to run some sort of chi-square test on the un-aggregated data. But more information about your situation would help.

There's many different kinds of matrices that can be obtained from data. Off the top of my head:

• a covariance matrix: you have several variables measured on each data point, and you want to see how they are correlated
• a misclassification matrix: you have a categorical output or response variable, and another variable that attempts to estimate or predict the response
• a grid of spatial measurements: adjacent rows/columns are in some sense closer to each other than to non-adjacent rows/columns
• a contingency table of counts or cell averages

The methods you'd use to analyse these matrices would differ. I'd guess that your matrix is the last one judging from the ANOVA mention, but it would be good to have more information.