Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
56

Here an example that can helps you: library(cluster) library(fpc) data(iris) dat <- iris[, -5] # without known classification # Kmeans clustre analysis clus <- kmeans(dat, centers=3) # Fig 01 plotcluster(dat, clus$cluster) # More complex clusplot(dat, clus$cluster, color=TRUE, shade=TRUE, labels=2, lines=0) # Fig 03 with(iris, pairs(dat, ...


56

Neural networks are widely used in unsupervised learning in order to learn better representations of the input data. For example, given a set of text documents, NN can learn a mapping from document to real-valued vector in such a way that resulting vectors are similar for documents with similar content, i.e. distance preserving. This can be achieved using, ...


55

Methods overview Short reference about some linkage methods of hierarchical agglomerative cluster analysis (HAC). Basic version of HAC algorithm is one generic; it amounts to updating, at each step, by the formula known as Lance-Williams formula, the proximities between the emergent (merged of two) cluster and all the other clusters (including singleton ...


37

In some sense I think this question is unanswerable. I say this because how well a particular unsupervised method performs will largely depend on why one is doing unsupervised learning in the first place, i.e., does the method perform well in the context of your end goal? Obviously this isn't completely true, people work on these problems and publish results ...


36

Imagine that you have a bunch of seeds fastened on a glass plate, which is resting horizontally on a table. Because of the way we typically think about space, it would be safe to say that these seeds live in a two-dimensional space, more or less, because each seed can be identified by the two numbers that give that seed's coordinates on the surface of the ...


32

Finding the differences can be done by looking at the models. Let's look at sparse coding first. Sparse coding Sparse coding minimizes the objective $$ \mathcal{L}_{\text{sc}} = \underbrace{||WH - X||_2^2}_{\text{reconstruction term}} + \underbrace{\lambda ||H||_1}_{\text{sparsity term}} $$ where $W$ is a matrix of bases, H is a matrix of codes and $X$ ...


27

I'd push the silhouette plot for this, because it's unlikely that you'll get much actionable information from pair plots when the number of dimension is 14. library(cluster) library(HSAUR) data(pottery) km <- kmeans(pottery,3) dissE <- daisy(pottery) dE2 <- dissE^2 sk2 <- silhouette(km$cl, dE2) plot(sk2) This approach is highly cited ...


26

Wang, Kaijun, Baijie Wang, and Liuqing Peng. "CVAP: Validation for cluster analyses." Data Science Journal 0 (2009): 0904220071.: To measure the quality of clustering results, there are two kinds of validity indices: external indices and internal indices. An external index is a measure of agreement between two partitions where the first ...


20

I've never encountered this term before. I am unsure whether it would spread light or darkness within either realm of statistics: those being machine learning (where supervised and unsupervised distinctions are central to problem solving) and inferential statistics (where regression, confirmatory analysis, and NHSTs are most often employed). Where those ...


19

This is called learning from positive and unlabeled data, or PU learning for short, and is an active niche of semi-supervised learning. Briefly, it is important to use the unlabeled data in the learning process as it yields significantly improved models over so-called single-class classifiers that are trained exclusively on known positives. Unlabeled data ...


17

A Distant supervision algorithm usually has the following steps: 1] It may have some labeled training data 2] It "has" access to a pool of unlabeled data 3] It has an operator that allows it to sample from this unlabeled data and label them and this operator is expected to be noisy in its labels 4] The algorithm then collectively utilizes the original ...


17

I routinely use $t$-SNE (alongside clustering techniques - more on this in the end) to recognise/assess the presence of clusters in my data. Unfortunately to my knowledge there is no standard way to choose the correct perplexity aside looking at the produced reduced dimension dataset and then assessing if it is meaningful. There are some general facts, eg. ...


15

I don't think I know more than you do, but the links you posted do suggest answers. I'll take http://www.cs.cornell.edu/~tomf/publications/supervised_kmeans-08.pdf as an example. Basically they state: 1) clustering depends on a distance. 2) successful use of k-means requires a carefully chosen distance. 3) Given training data in the form of sets of items ...


15

You want to look into self-organizing maps. Kohonen (who invented them) wrote a book about them. There are packages for this in R (som, kohonen), and there are implementations in other languages such as MATLAB.


15

Great question. Anything can be good or bad, useful or not, based on what your goals are (and perhaps on the nature of your situation). For the most part, these methods are designed to satisfy different goals. Statistical tests, like the $t$-test allow you to test scientific hypotheses. They are often used for other purposes (because people just aren't ...


14

If you randomly split the sample into 5 subsamples your 5 means will almost coincide. What is the sense of making such close points the initial cluster centres? In many K-means implementations, the default selection of initial cluster centres is based on the opposite idea: to find the 5 points which are most far apart and make them the initial centres. You ...


12

There are some internal clustering methods. In particular with respect to the distances of objects in the data set. See for example Silhouette coefficient [on Wikipedia]. You must however be aware that there are algorithms such as k-means that try to optimize exactly these parameters, and as such you introduce a particular type of bias; essentially this is ...


12

Boosting (as mentioned in the linked discussion) is a method that combines a set of algorithms to get a result that is better than what you can get from any single algorithm. For example random forests is a method for combining various classification trees for a classification algorithm. This approach is formally called ensemble averaging (although the ...


12

The only application of cross-validation to clustering I know of is this one: Divide the sample into a 4 parts training set & 1 part testing set. Apply your clustering method to the training set. Apply it also to the test set. Use the results from Step 2 to assign each observation in the testing set to a training set cluster (e.g. the nearest centroid ...


11

How about anomaly detection algorithms? As you mention Andrew Ng's class you'd probably seen the "XV. ANOMALY DETECTION" section on ml-class.org, but anyway. Anomaly detection will be superior to a supervised classification in scenarios similar to yours because: normally you have very few anomalies (ie., too little "positive" examples) normally you have ...


11

It's actually possible to get probabilities out of a Support Vector Machine, which might be more useful and interpretable than an arbitrary "score" value. There are a few approaches for doing this: one reasonable place to start is Platt (1999). Most SVM packages/libraries implement something like this (for example, the -b 1 option causes LibSVM to produce ...


11

Whereas $k$-means tries to optimize a global goal (variance of the clusters) and achieves a local optimum, agglomerative hierarchical clustering aims at finding the best step at each cluster fusion (greedy algorithm) which is done exactly but resulting in a potentially suboptimal solution. One should use hierarchical clustering when underlying data has a ...


11

Scalability $k$ means is the clear winner here. $O(n\cdot k\cdot d\cdot i)$ is much better than the $O(n^3 d)$ (in a few cases $O(n^2 d)$) scalability of hierarchical clustering because usually both $k$ and $i$ and $d$ are small (unfortunately, $i$ tends to grow with $n$, so $O(n)$ does not usually hold). Also, memory consumption is linear, as opposed to ...


11

Not going to address clustering because it's been addressed in other answers, but: In general, the problem of testing whether two samples are meaningfully different is known as two-sample testing. By doing a $t$-test, you severely limit the kinds of differences that you're looking for (differences in means between normal distributions). There are other ...


11

I'm going to answer this as an approach to clustering categorical data. The standard k-means performs poorly in case of categorical data since in the sample space is discrete. The cost function defined by k-means computes the Euclidean Distance (or something similar) which is relevant only for continuous variables. Instead of computing the Euclidean ...


11

I'll borrow the notation from (1), which describes GMMs quite nicely in my opinon. Suppose we have a feature $X \in \mathbb{R}^d$. To model the distribution of $X$ we can fit a GMM of the form $$f(x)=\sum_{m=1}^{M} \alpha_m \phi(x;\mu_m;\Sigma_m)$$ with $M$ the number of components in the mixture, $\alpha_m$ the mixture weight of the $m$-th component and $...


10

An outline of internal clustering criteria (internal cluster validation indices) This is the excerpt from my documentation of a number of popular internal clustering criteria I've programmed, as a user, for SPSS Statistics (see my web page). 1. Reflections Internal clustering criteria or indices exist to assess internal validity of a partition of objects ...


10

http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icml04.icdm06long.pdf Some papers to help you further understand what blending is. I think you can also google for ensemble selection/learning, and stacking as well. Your general understanding of 'mixing up outcomes from many models and resulting in a better result' is correct though.


10

In neuroscience the term Neural Coding is used to refer to the patterns of electrical activity of neurons induced by a stimulus. Sparse Coding by its turn is one kind of pattern. A code is said to be sparse when a stimulus (like an image) provokes the activation of just a relatively small number of neurons, that combined represent it in a sparse way. In ...


9

Do I understand you correctly that you want to measure whether C1 is a faster/slower learner than C2? With unlimited training data, I'd definitively construct (measure) the learning curves. That allows you to discuss both questions you pose. As Dikran already hints, the learning curve does have a variance as well as a bias component: training on smaller ...


Only top voted, non community-wiki answers of a minimum length are eligible