Assumptions of cluster analysis

Question

Apologies for the rudimentary question, I am new to this form of analysis and have a very limited understanding of the principles so far.

I was just wondering if many of the parametric assumptions for multivariate/univariate tests apply for Cluster analysis? Many of the sources of information I have read regarding cluster analysis fail to specify any assumptions.

I am particularly interested in the assumption of independence of observations. My understanding is that violation of this assumption (in ANOVA and MAVOVA for example) is serious because it influences estimates of error. From my reading so far, it seems that cluster analysis is largely a descriptive technique (that only involves statistical inference in certain specified cases). Accordingly, are assumptions such as independence and normally distributed data required?

Any recommendations of texts that discuss this issue would be greatly appreciated. Many Thanks.

Answer 1

Well, clustering techniques are not limited to distance-based methods where we seek groups of statistical units that are unusually close to each other, in a geometrical sense. There're also a range of techniques relying on density (clusters are seen as "regions" in the feature space) or probability distribution.

The latter case is also know as model-based clustering; psychometricians use the term Latent Profile Analysis to denote this specific case of Finite Mixture Model, where we assume that the population is composed of different unobserved groups, or latent classes, and that the joint density of all manifest variables is a mixture of this class-specific density. Good implementation are available in the Mclust package or Mplus software. Different class-invariant covariance matrices can be used (in fact, Mclust uses the BIC criterion to select the optimal one while varying the number of clusters).

The standard Latent Class Model also makes the assumption that observed data come from a mixture of g multivariate multinomial distributions. A good overview is available in Model-based cluster analysis: a Defence, by Gilles Celeux.

Inasmuch these methods rely on distributional assumptions, this also render possible to use formal tests or goodness-of-fit indices to decide about the number of clusters or classes, which remains a difficult problem in distance-based cluster analysis, but see the following articles that discussed this issue:

1. Handl, J., Knowles, J., and Kell, D.B. (2005). Computational cluster validation in post-genomic data analysis. Bioinformatics, 21(15), 3201-3212.
2. Hennig, C. (2007) Cluster-wise assessment of cluster stability. Computational Statistics and Data Analysis, 52, 258-271.
3. Hennig, C. (2008) Dissolution point and isolation robustness: robustness criteria for general cluster analysis methods. Journal of Multivariate Analysis, 99, 1154-1176.

Answer 2

There is a very wide variety of clustering methods, which are exploratory by nature, and I do not think that any of them, whether hierarchical or partition-based, relies on the kind of assumptions that one has to meet for analysing variance.

Having a look at the [MV] documentation in Stata to answer your question, I found this amusing quote at page 85:

Although some have said that there are as many cluster-analysis methods as there are people performing cluster analysis. This is a gross understatement! There exist infinitely more ways to perform a cluster analysis than people who perform them.

In that context, I doubt that there are any assumptions applying across clustering method. The rest of the text just sets out as a general rule that you need some form of "dissimilarity measure", which need not even be a metric distance, to create clusters.

There is one exception, though, which is when you are clustering observations as part of a post-estimation analysis. In Stata, the vce command comes with the following warning, at page 86 of the same source:

If you are familiar with Stata’s large array of estimation commands, be careful to distinguish between cluster analysis (the cluster command) and the vce(cluster clustvar) option allowed with many estimation commands. Cluster analysis finds groups in data. The vce(cluster clustvar) option allowed with various estimation commands indicates that the observations are independent across the groups defined by the option but are not necessarily independent within those groups. A grouping variable produced by the cluster command will seldom satisfy the assumption behind the use of the vce(cluster clustvar) option.

Based on that, I would assume that independent observations are not required outside of that particular case. Intuitively, I would add that cluster analysis might even be used to the precise purpose of exploring the extent to which the observations are independent or not.

I'll finish by mentioning that, at page 356 of Statistics with Stata, Lawrence Hamilton mentions standardized variables as an "essential" aspect of cluster analysis, although he does not go into more depth on the issue.

Answer 3

Spatial cluster analysis uses geographically referenced observations and is a subset of cluster analysis that is not limited to exploratory analysis.

Example 1

It can be used to make fair election districts.

Example 2

Local spatial autocorrelation measures are used in the AMOEBA method of clustering. Aldstadt and Getis use the resulting clusters to create a spatial weights matrix that can be specified in spatial regressions to test a hypothesis.

See Aldstadt, Jared and Arthur Getis (2006) “Using AMOEBA to create a spatial weights matrix and identify spatial clusters.” Geographical Analysis 38(4) 327-343

Example 3

Cluster analysis based on randomly growing regions given a set of criteria could be used as a probabilistic method to indicate unfairness in the design of institutional zones such as school attendance zones or election districts.

Answer 4

Cluster analysis does not involve hypothesis testing per se, but is really just a collection of different similarity algorithms for exploratory analysis. You can force hypothesis testing somewhat but the results are often inconsistent, since cluster changes are very sensitive to changes in parameters.

http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_introclus_sect010.htm