The notification after the ANOVA table after K-means analysis indicates that significance levels should not be looked at as the test of equal means, as the cluster solution has been derived based on Euclidean distance to maximize the distance. What test should I use to show whether the means of the clustering variables differ among the clusters? I have seen this warning in k-means outputs' provided ANOVA table, but in some references I see that post-hoc ANOVA tests are run. Should I ignore k-mean ANOVA outputs and run one-way ANOVA with post-hoc tests and interpret them in a traditional way? Or can I only imply about magnitude of F value and which variables contributed more to difference? Another confusion is that clustering variables are not normally distributed violating assumption of ANOVA, then I could use Kruskal-Wallis non-parametric test, but it has assumption about the same distributions. The inter-cluster distributions for the specific variables do not seem the same, some are positively skewed, some are negatively... I have 1275 large sample, 5 clusters, 10 clustering variables measured in PCA scores.
You must not use the same data to 1) perform clustering and 2) hunt for significant differences between the points in the clusters. Even if there's no actual structure in the data, the clustering will impose one by grouping together points which are nearby. This shrinks the within-group variance and grows the across-group variance, which biases you towards false positives.
This effect is surprisingly strong. Here are the results of a simulation that draws a 1000 data points from a standard normal distribution. If we assign the points to one of five groups at random before running the ANOVA, we find that the p-values are uniformly distributed: 5% of the runs are significant at the (uncorrected) 0.05 level, 1% at the 0.01 level, etc. In other words, there is no effect. However, if $k$-means is used to cluster the data into 5 groups, we find a significant effect virtually every time, even though the data has no actual structure.
There is nothing special about an ANOVA here--you would see similar effects using non-parametric tests, logistic regression, anything. In general, validating a clustering algorithm's performance is tricky, particularly if the data are not labelled. However, there are a few approaches to "internal validation", or measuring the clusters' quality without using external data sources. They generally focus on the compactness and separability of the clusters. This review by Lui et al. (2010) might be a good place to start.
Your real problem is data snooping. You can't apply ANOVA or KW if the observations were assigned to groups (clusters) based on the input data set itself. What you can do is to use something like Gap statistic to estimate the number of clusters.
On the other hand, the snooped p-values are biased downward, so if ANOVA or KW test result is insignificant, then the "true" p-value is even larger and you may decide to merge the clusters.
I think you could apply such an approach (i.e. using the statistics, such as F-statistics or t-statistics or whatever), if you toss out the usual null distributions.
What you'd need to do is simulate from the situation in which your null is true, apply the whole procedure (clustering, etc), and then calculate whichever statistic each time. Applied over many simulations, you would get a distribution for the statistic under the null against which your sample value could be compared. By incorporating the data-snooping into the calculation you account for its effect.
[Alternatively one could perhaps develop a resampling-based test (whether based on permutation/randomization or bootstrapping).]