Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

I have records of census data comprised of complex data types including distributions, such as these two fields from a record, both of which are distributions.

The first is an income distribution:

No income: 1110.0               
$1 to $9,999 or loss: 13840.0   ######
$10,000 to $14,999: 9490.0      ####
$15,000 to $24,999: 152145.0    #######################################################################
$25,000 to $34,999: 5465.0      ##
$35,000 to $49,999: 4950.0      ##
$50,000 to $64,999: 6880.0      ###
$65,000 to $74,999: 10420.0     ####
$75,000 or more: 9100.0         ####

And the second is an age distribution:

< 5: 955.0        #####
5 - 9: 1002.0     #####
10 - 14: 1032.0   #####
15 - 19: 1183.0   ######
20 - 24: 1305.0   #######
25 - 29: 1082.0   ######
30 - 34: 1212.0   ######
35 - 39: 1175.0   ######
40 - 44: 1210.0   ######
45 - 49: 1359.0   #######
50 - 54: 1336.0   #######
55 - 59: 1212.0   ######
60 - 64: 1034.0   #####
65 - 69: 727.0    ####
70 - 74: 618.0    ###
74 - 79: 501.0    ##
80 - 84: 404.0    ##

A full example record can be seen at http://tinypic.com/r/34s2kxj/6

Now what I need to do is create a correlation coefficient matrix for factor analysis. But the correlation matrix needs show correlations between fields in the full records. For example I'd like a correlation coefficient that describes the relationship between age and income, using these two distributions.

Where I'm stumped is how to calculate correlation coefficients for two distributions. For example, if I want to find the correlation coefficient for age distribution and income distribution.

I don't have access to the original data that makes up these distributions, only the summarized data that you see.

Any ideas? Right now I'm investigating using a Q-Q Plot (but I'm having trouble figuring out how to generate an intelligent coefficient from it), a cross-correlation (but I'm not sure a wave function is applicable to this type of data), Ripley's Cross-K function (which seems to be only applicable to temporal data), and fabricating matrices to overlay on the histogram and then calculating correlations between the cells. I've also considered calculating correlation coefficients of the moments of the distribution (mean, skew, kurtosis, std dev), but I've never heard of this and I'm not sure that it's valid.

Any help or a nudge in the right direction would be much appreciated!

share|improve this question
3  
When all you have are the marginal distributions, as shown here, you have absolutely no information about their correlation. You could simulate a population having these two distributions. Then you could pair ages and incomes by taking the largest with the largest, the second largest with the second largest, and so on, creating a high positive correlation. You could equally well pair the largest income with the smallest age, etc., creating a high negative correlation. Your data say nothing about what pairing actually occurs. – whuber♦ Jul 30 '12 at 16:16
That makes sense to me when looking at two instances of these distributions. However, keep in mind that this is a single record. Doesn't the process of calculating a correlation coefficient only make sense using a large sample population? You wouldn't try to correlate two numbers using only a single data point either. But, over, say 500 of these records, doesn't it make sense that you could find a pattern? – Jason Kolb Jul 30 '12 at 16:21
For example, using just the moments of the distribution such as the mean, couldn't you at the very least find correlations between moments? For example if, instead of the distribution data itself I simply used the mean of the two distributions, and then calculated the Pearson coefficient normally against the means, would that not give me information about how the mean of one distribution correlates to the mean of the other? – Jason Kolb Jul 30 '12 at 16:22
There seems to be some confusion in these comments, Jason. You appear now to be referring to a collection of about 500 joint distributions, not a single distribution. Your original question refers to "correlation" in the sense of a "relationship between age and income": that is, the correlation of a single distribution (for a single record). These relationships could change from one record to another. Now, in the comments, you write of a different correlation, one that measures associations revealed by all 500 records. The two things are not the same! Which one are you really after? – whuber♦ Jul 30 '12 at 16:33
1  
Confounding these two kinds of correlations is known as the ecological fallacy: perhaps that article is a useful nudge? – whuber♦ Jul 30 '12 at 16:38
show 4 more comments

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.