2
$\begingroup$

I am new to machine learning. I am reading the papers K-means Clustering via Principal Component Analysis and PCA-guided search for K-means. But there are too many mathematical proofs in these papers. I can't understand these papers easily. Can anyone explain this thing in simple words?

Also, I am trying to use Python to experiment with this approach. But the result is far form the paper.

My code is below. I use the The AT&T Face Data Set dataset.

TRAIN_PEOPLE = 40
MAX_ITER = 1000
# k-means starting

print("method       Sum of distances")
pca = PCA(n_components = TRAIN_PEOPLE).fit(X)
pca_result = KMeans(n_clusters=TRAIN_PEOPLE, max_iter = MAX_ITER).fit(pca.transform(X))
kmeans = KMeans(init=pca_result.cluster_centers_.dot(pca.components_), 
     n_clusters=TRAIN_PEOPLE, n_init=1, max_iter = MAX_ITER).fit(X)

print("pca-guided   " + str(kmeans.inertia_))

kmeans = KMeans(init='k-means++', n_clusters=TRAIN_PEOPLE, n_init=1, max_iter = MAX_ITER).fit(X)
print("k-means++    " + str(kmeans.inertia_))


kmeans = KMeans(init='random', n_clusters=TRAIN_PEOPLE, n_init=1, max_iter = MAX_ITER).fit(X)
print("random       " + str(kmeans.inertia_))

Here is the result:

method       Sum of distances
pca-guided   214982166.562   // too high!
k-means++    161294842.543
random       170072750.47

Could anyone explain what's going on here?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Re the first paper see here stats.stackexchange.com/questions/183236. Does it help? $\endgroup$
    – amoeba
    Commented Sep 29, 2016 at 8:16
  • $\begingroup$ @amoeba yes.But why the experiment result is wrong ? $\endgroup$
    – Fanny ZMN
    Commented Sep 29, 2016 at 11:58
  • $\begingroup$ k-means is randomized. Maybe you were just unluckym Try multiple runs! $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Sep 29, 2016 at 19:16
  • $\begingroup$ yes.I tried.the same result. $\endgroup$
    – Fanny ZMN
    Commented Sep 30, 2016 at 5:01
  • $\begingroup$ When you transform cluster centroids in the PCA space to the original space, don't you forget to add the variable means? $\endgroup$
    – amoeba
    Commented Sep 30, 2016 at 23:07

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.