Unsupervised clustering of households into types

Question

Traditionally, households fall into a couple of discrete categories. For example:

1. Husband and wife
2. Husband, wife and young kids
3. Divorced Wife and kids
4. Bachelor
5. Adult child living with husband and wife

I have a set of data describing households in Southern Africa where nuclear families are much more rare. I have about 2000 households and demographic details such as age, gender, marital status, education and so on for all household members (in some cases there are up to 23 people in one house).

My question is how would I go about grouping these households into clusters in a way that would allow me to describe the most common household membership patterns (as I did for nuclear families above).

I tried combining all demographic details into a single code and discovered that 51% of all 14 157 people fall into one of 4 categories - 3197(23%) are Male/Unemployed/Present @ home most nights/ None to 7yrs education/ age 0-18yrs / marital status Single. 3074(22%) Female_Unemployed_Present_None-G7_0-18yrs_Single, etc. But this doesn't get to my question about how these people are configured together in a home.

I also tried using this combined demographic variable and household membership (i.e. 3 people all live together and so have the same household code) in a cluster analysis but did not get usable results.

Any ideas would be gratefully received.

Answer 1

Most (but not all) clustering algorithms will partition your data set into disjoint, non-overlapping clusters that cover the entire data set. But there will likely be uncommon "configurations" that probably do not fit into any cluster.

Furthermore, you will need to define a suitable similarity measure. When are two households similar? This is domething you need to decide, because there is no mathematical "correct" way. E.g. you could look at the average number of attributes every member has in common to the most similar member in the other household. So e.g. two persons in each household, and one has 4/5 attributes in common, the other 5/5. Then you could define the similarity to be 9/10 i.e. 90%.

An alternate, more promising, approach is that of frequent itemset mining, which yields common patterns but does not require every data point to conform to exactly one of them. Thr classic use case is "people who bought X also bought Y". It is possible (but not a matter of point & click) to abstract this to "households where a person with attributes X live, also contain persons with attributes Y and Z", along woth frequency and likelihood values.