How to choose a fair gamma value when performing k-prototypes clustering?

Question

In the k-prototypes clustering algorithm, the distance function consists of two dissimilarity components - one for the numerical elements of the observations, and one for their categorical elements. In pseudo-mathematical terms, we have:

$$d(x, y) = d_1(x_{num}, y_{num}) + \gamma d_2(x_{cat}, y_{cat})$$

where $\gamma > 0$ is a weight that allows to favor more or less one of the two data types when building centroids. (Setting $\gamma$=0 boils down to performing clustering on numerical variables only, whereas the greater the weight, the greater the impact of categorical variables on the final result.)

In his introduction to k-modes and k-prototypes from 1998, Huang says the following: "Average standard deviation of numeric attributes may be used as a guidance in specifying $\gamma$. [...] However, it is too early to consider this as a general rule."

Have some clear mathematical results been found since then? How can I pick a $\gamma$ value so that my clustering procedure doesn't favor one data type over the other?

Answer 1

As you already mentioned, and according to the kmodes library, the standard deviation of numeric variables is calculated using the mean:

if gamma is None:
    gamma = 0.5 * np.mean(Xnum.std(axis=0))

This is in accordance with Huang [1997]

See this.

---

Reference:

Huang, Z.X. (1997) Clustering Large Data Sets with Mixed Numeric and Categorical Values. Proceedings of the First Pacific Asia Knowledge Discovery and Data Mining Conference, Singapore, World Scientific, 21-34.

Answer 2

Note that when the numerical features are scaled (z = (x-u)/s, where u is the mean and s the standard deviation), the default gamma value that is calculated will be 0.5 which likely will give very low weight to the categorical dissimilarity.

In my dataset, the default gamma value of 0.5 is almost equivalent to ignoring the categorical features. (See the statistics of my dataset sample below)

 Categorical dissimilarity stats: min=0, max=6, mean=3, median=3.0
 Numerical dissimilarity stats: min=0.0, max=11082.115119330787, mean=41.10081233373842, median=15.116343318087967

So I am using a gamma of 10 to give categorical feature more weight.