Constructing features from k-means I would like to construct features from the result of apply k-means clustering to construct features of my data that I can later use for a classifier.
Assume I have fixed the $k$ (e.g. 5) and performed the clustering. Then I could use 


*

*the indicator of the cluster

*the center of the cluster


as new features. What else is done in practice? I read about using the distance to the cluster centers as feature - can we find any reference for this?
 A: There is a great paper relating k-means with sparse coding of features, and how to address some of its weaknesses to produce good features. Even through it is focused on the particular case of image processing, it has valuable advice for the general case (how to do prewhitening to decorrelate data and so on).
Finally, it is well known that algorithms like kmeans and knn which (in its original formulation) use euclidean distance as a metric, perform poorly in a high dimensional setting. Here there is an importance reference addressing this point.
Edit: I came across this (IMHO really interesting) paper Deterministic Feature Selection for k-Means Clustering, which provides a deterministic algorithms with theoretical analysis and performance warranties. See also some of the references therein, specially those by the first author.
Just to make one thing clear: what is the problem you are addressing (number of samples, dimensionality, etc)?. The motivation for feature selection in this paper is the poor performance of k-means in high dimensional space.
Often one assumption is made: only a few of the many features are relevant. Many approaches are suboptimal in some way: like greedy search and randomized search, and not all have warranties on their performance.
So what you do is iterate over a number of trials/alternative heuristics until you find a satisfactory result.
So in case you need to build new features, you could try to generate new, sensible features and then perform feature selection on whole set of features.
Hope this helps.
A: Basically there is no you must add this or this.....you actually should add features that verifiable improve your classification. 
You can be creative here and try several things.
E.g. you can create statistics for variables (mean, sd,...) for each cluster and add these. You can also then add the difference to this new mean / median /... for example. 
Also adding cluster 'quality measures' might be an idea, like
Intra cluster distance for each cluster,....
You can also try different clustering methods to create additional features.
Keep in mind, just creating these variables is not everything, you also have to check if your classification improves.
From my own experience:
Most of the times I could not improve classification results with new features created by clustering (but this is of course highly dataset dependent)
Another important thing: 
Make sure you do not include the target variable of your later on testset for classification in the clustering. This will give misleading results of classification performance.
