Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0
votes
1answer
478 views

How to validate k-means result [duplicate]

I'm doing anomaly detection on unsupervised data using k-means I got a result but I don't know how to validate my clustering result. by plotting I can see my anomalies but how should I validate that ...
0
votes
1answer
272 views

Threshold for kmeans anomaly detection

I'm learning the kmeans to find out anomaly from the dataset. but I don't know how to set threshold. I tried by the putting mean of the centroid to point distance but it's not working, half my record ...
0
votes
1answer
3k views

how to handle sparse data problem in unsupervised learning .i'm going to use k means on data set

how to handle sparse data problem in unsupervised learning .i'm going to use k-means on the dataset. I have 200 variables, nearly in each column have 70% zeros. how can I handle without discarding any ...
0
votes
1answer
47 views

How to improve classification algorithms when the features are unlabeled?

hey guys I'm dealing with data that has around 15 features that are all positive, real numbers but I have no idea what the features are. The headers of my data are all just "feature1", "feature2", "...
0
votes
1answer
278 views

What is a good reason why reinforcement learning relies on Q value instead of the reward r?

I am new to reinforcement learning and I am reading off of some tutorial materials, but I have noticed a seldom discussed assumption that we should calculate our action based on Q-value instead of the ...
0
votes
0answers
62 views

How can I classify my text by using unsupervised approaches?

I have a corpus of newspaper articles and would like to classify them according to content (words, phrases) that I determine beforehand. The only way to do this that I found so far is by looking at ...
0
votes
1answer
56 views

Learning vs. training

In my head, these two words (learning and training) seem to somehow have a fuzzy boundary between them. For example, the word learning for me conveys the idea of training; if I want to learn ...
1
vote
0answers
33 views

How can we learn the values of the parameters for Levenshtein distance?

I am trying to filter out similar-looking names from a database. Once I have figured out the names, I will merge them into a single entity. To achieve this, I am planning on using edit-distance ...
2
votes
1answer
2k views

how to handle missing data in clustering problem

The features in data sometimes contains missing values, which mean the value is unknown. If I replace unknown value with a special normal value like "0", then the clustering algorithms will trade them ...
0
votes
1answer
35 views

What is the best way to analyze and predict based on a dataset that has both text and numbers?

I have a Twitter dataset that has both the tweets themselves, but also metadata about the user as number of tweets, date of creation, and other numerical values. How can I make predictions based on ...
2
votes
0answers
54 views

How can knowing test data improve the performance of face recognition?

I don't think knowing face recognition is necessary to answer this question, any suggestions are welcome. For face verification problems, usually images are first converted into fixed length vectors ...
1
vote
1answer
37 views

Improving clusters

Say we have 10 classes. We have obtained two kinds of clusters in an unsupervised manner. One type is heavily clustered, i.e, say it shows only 4 clusters and one heavily cut, i.e, it shows say, 20 ...
5
votes
5answers
520 views

Choosing a model for my unsupervised machine learning problem

I need to choose a model for unsupervised machine learning problem. There are 4 clusters in 3D space. These are my requirements: I will run the same model multiple times with different training data (...
0
votes
1answer
165 views

Resulting clusters are very uneven

I am clustering a set of 50k products. I would expect the resulting clusters to be things like "organic chicken", "orange juice", etc. I am using the bag of words model, and there are about 8k ...
0
votes
1answer
317 views

Cluster data based on similarity, but split by one feature (Python, R)

Is there a way to do unsupervised clustering based on similarity (like all other methods), but create clusters by splitting on just one (or specific) features? For example, I have customer data with ...
2
votes
1answer
101 views

Any way to recognize pattern(such as Char and Number) from image without labeled data?

I am trying to build a captcha recognizer. I found CNN play very well if there are enough labeled data. For example, I use this https://github.com/lepture/captcha to generate 4 size char+number ...
2
votes
1answer
2k views

Which unsupervised classification method can be used for categorical data?

I have a list of categorical data and I want to apply an unsupervised classification method to cluster this data. Which method could be used? Example: gene1 gene2 gene3 gene4 ...
1
vote
1answer
251 views

What is the difference between SVI and online HDP algorithms?

I'm going over the paper by Hoffman et. al. titled "Stochastic Variational Inference" (SVI) where the authors present an algorithm for Hierarchical Dirichlet Process (HDP) topic models. I'm comparing ...
1
vote
1answer
461 views

clustering for categorical data with one column for observations

I'm trying to cluster a dataset using 4 variables, all of which are categorical variables. I'd also like to include another numerical variable that's actually the number of observations of another ...
1
vote
1answer
133 views

Objective performance measures for PCA?

How can I measure how good my PCA was ? Of course, it depends A LOT on context, or even maybe subjectivity sometimes. But what objective measures can I use to measure how good my PCA was? It doesn't ...
4
votes
1answer
2k views

SOM grid size suggested by Vesanto

I am a bit confused on the size of the SOM grid size suggested by Vesanto. Here in this link, it says 5*sqrt(N) where N is mentioned as the dataset size. What is ...
0
votes
1answer
67 views

Branches or taxonomy of dimensionality reduction?

I need a high-level taxonomy of dimensionality reduction approaches. I need your help to unify what I have found: 1) according to wikipedia, dimensionality reduction can be divided into: feature ...
0
votes
1answer
40 views

Can we say that clustering is a function?

Can we say that clustering is a function? Of course, the term "clustering" is broad and can mean different things in different contexts: clustering as an area in machine learning, clustering ...
1
vote
1answer
240 views

What is the typical taxonomy for clustering methods?

What is the typical taxonomy for clustering methods? For example, for regression we can talk about: simple regression, multiple univariate regression, and multivariate regression. And then, we can ...
5
votes
1answer
2k views

Generative models for time series simulation

I have a basic understanding of generative models, like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). It seems that they are mostly used to in the field of image ...
2
votes
0answers
284 views

Metric for unsupervised recommender-system competition?

I have a data source containing millions of documents from a wide variety of business domains. We've aggregated the data such that we can easily find information using natural-language search queries. ...
0
votes
0answers
63 views

“Uniform” Clustering

I have the data $\{(y_i,\omega_i), i=1,\dots,n\}$, where $y_i \in \mathbb{R}_+$ the response and $\omega_i>0$ a weight. Fixed $K>0$ I want to determinate $y_1^*,\dots,y_{K-1}^*$ such that $$ \...
1
vote
0answers
163 views

Improve gaussian mixture model performance

I have a data set of 10-dimensional cell measurements for leukemia. The data points are unlabeled and the task is to find the ratio of pathological measurements w.r.t. the rest of the sample. In other ...
-2
votes
1answer
294 views

Grouping data based on a specific pattern [closed]

I have a collection of data for a multiplayer game (2000 games, 10 players each). I would like to create clusters from this data, each containing the ids of 3 players that had played against each ...
0
votes
0answers
168 views

How do i display the first principal component of an image after doing PCA (using SVD)?

Suppose i have an 10 images with 100x100 pixels. I have already converted the data into a 10x10000 dataset, subtracted the mean and performed SVD to get the eigenvectors and eigenvalues. Now i want to ...
3
votes
1answer
259 views

Assumptions on class priors in expectation-maximization?

I want to use the EM algorithm to do clustering under a missing labels regime. The assumption I am making about the missing data is that it's distributed according to Bernoulli distribution. So for ...
0
votes
3answers
273 views

Detect and remove abnormal data from 1D data

I have a 1D data which represents order values. Sometimes a client creates a test order which they don't remove from the system. What would be the best way of detecting and removing values that seem ...
0
votes
0answers
16 views

Which machine learning algorithm may help to explore the importance of contributions of variables to their sum over time?

I have data in the form of a $n \times t$ matrix $X$ where $n$ is a number of variables and $t$ a (large) number of time points. At any given time point the elements of the matrix can be expressed as ...
8
votes
3answers
11k views

Do we need to set training set and testing set for clustering?

When we do classification and regression, we usually set testing and training sets to help us build and improve models. However, when we do clustering do we also need to set testing and training sets?...
2
votes
1answer
240 views

Empty nodes when creating SOM

I am trying to create a SOM map based on records with different discrete classifications (tags) like the example below ...
0
votes
0answers
80 views

What unsupervised methods are used for sampling a dataset?

I have a dataset of 21 classes. Each class has different number of samples. I want to sample an specific number of samples from each class. For example 50. I thought of using K-means to cluster 50 ...
-1
votes
2answers
2k views

Support Vector Machine - Classification or Clustering

I don't really understand if SVM are classification methods (like Logistic regression) or clustering methods. Since it's used for supervised learning, it should be part of classification methods ...
0
votes
1answer
593 views

Comparing supervised text classification algorithms with unlabeled documents from web

Working with the unlabeled documents from web for supervised text classification, even though the problem settings dictates using semi-supervised learning, I aim to compare several different ...
1
vote
2answers
1k views

Constrained Clustering Implementation R or Python? [closed]

Can anyone point me to an implementation in R or Python of a constrained clustering algorithm? In case this is overly broad, I am hoping to exploit known must-link/cannot-link pairs to improve the ...
1
vote
0answers
58 views

Supervised learning for audio files with uncertain labels

There exists a collection of audio files, each 30 seconds long. Such an audio file contains the recording of the manufacturing of a certain small piece in a machine. After manufacturing the piece, it ...
1
vote
0answers
38 views

How to check the quality of clustering results if there is no labels? [duplicate]

How can we check the quality of clustering of no-labeled data? I learnt from class that there are some ways to achieve this . One is to "measure intra-cluster cohesion (how near the data points ...
2
votes
2answers
909 views

Unsupervised Topic Models that don't require the number of topics to be set upfront

I hope this question isn't too general, but I'm looking for an unsupervised topic modelling algorithm that doesn't require the number of topics (k) to be defined prior to running the analysis of the ...
0
votes
1answer
178 views

Multiple correspondence analysis for clustering (unsupervised learning) [closed]

I have limited stat/coding knowledge yet I try to do user clustering using unsupervised method using R. I have about 2795 observations gained from survey (mixture of categorical and scale questions). ...
1
vote
0answers
482 views

Finding brief repeated patterns in a time series

Suppose I have a series of events, each of which is an element of {-1, 0 1} with a known distribution. Events happen in approximately continuous time. My dependent variable is the sum of the series. ...
1
vote
0answers
54 views

Does it make sense to use auto-encoders to reconstruct GIST features?

I am trying to extract good low dimensional representation of CIFAR-10 images in an unsupervised way. It is a project requirement that I use 512-d GIST features, reduce the dimensions to 32 using PCA ...
1
vote
1answer
946 views

Classification using lookup table

I have a matrix of samples to classify, samples are matrix columns and features (noisy or estimated features) are matrix rows. On the other hand, I have a lookup table for correspondence between ...
0
votes
1answer
1k views

Do RNNs (recurrent neural networks) support feature learning or not?

Please explain how RNNs support feature learning problem of deep learning or why it does not support it. How about unsupervised learning; Does RNN support that?
29
votes
5answers
5k views

Distinguishing between two groups in statistics and machine learning: hypothesis test vs. classification vs. clustering

Assume I have two data groups, labeled A and B (each containing e.g. 200 samples and 1 feature), and I want to know if they are different. I could: a) perform a statistical test (e.g. t-test) to see ...
0
votes
1answer
2k views

How to perform Validation on Unsupervised learning?

Since I consider Unsupervised learning, I don't have any ground truth to compare with, during the validation phase. So, is there any standard method to deal with it? Additional informations: in my ...
1
vote
0answers
140 views

Performance measure for iterative semi-supervised learning

Consider the problem of semi-supervised learning where, in each round, the labels of all data points are guessed and then the label of a random data point is revealed. As the labels of more and more ...