Questions tagged [unsupervised-learning]
Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.
0
votes
0answers
11 views
Clustering as a method to find and label classes for supervised learning
I'm working on a text classification project. We have around 300k documents (small, 1~2 phrases) and we don't know the set of labels or how many labels there are. The recommended approach to me was ...
0
votes
1answer
10 views
Latent variable in Gaussian Mixture Model
Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
0
votes
2answers
53 views
Why are decision trees considered supervised learning?
It seems to work similar to clustering algorithms, where data does not have to be labeled, and the algorithm creates it's own labels/groups based on feature similarities...
0
votes
0answers
10 views
clustering objects in point cloud
I am currently working on point cloud data analysis, trying to label objects which are not ground or vegetation e.t.c. So far I tried many clustering algorithms, with moderate success.
In my best ...
0
votes
0answers
10 views
Shadow significance
I've faced specific issue recently and kindly ask you to help.
Imagine standard linear supervised learning framing for binary classification problem (X,y, OLS, p-vals, etc.).
One can develop common ...
0
votes
0answers
10 views
DBSCAN loops one or several times a data point?
I am trying to construct a model data to facilitate the clustering algortihms execution in terms of searching for data point in the dataset. This model is a set of connections between points such that ...
0
votes
0answers
21 views
Clustering on the basis of user behaviour for a particular app
I have an app data for different customer. The columns are all the functionalities in the app and rows are all 1s and 0s. This dataset has 70k rows with most of them 0s i.e. functionality not being ...
3
votes
2answers
45 views
External validation of clustering requires labels, but why cluster at all if you have labels?
There are two types of validation in clustering, using:
Internal indexes: Used to measure the goodness of a clustering structure without respect to external information (e.g., sum of squared errors)...
1
vote
1answer
18 views
How can I tell if an Autoencoder is encoding my data properly?
Autoencoders can be classified as a method of unsupervised learning, and unsupervised learning often comes with a problem where it's hard to tell if the model is working properly.
However, some ...
1
vote
0answers
26 views
Clustering Noisy Data [closed]
There are various ways to cluster data. Some require the data first to be scaled to have a mean of $0$ and standard deviation of $1$. However, others do not mention if the data should be scaled at all....
0
votes
1answer
26 views
Is clustering subsequences of time-series still meaningless with unsupervised learning?
In the paper "Clustering of Time Series Subsequences Is Meaningless" Keoh et al. claim that breaking a time-series into chunks (sometimes called lags) of fixed-size using the rolling window method ...
0
votes
0answers
59 views
Predict document topic with LDA using only document-topic probabilities
I did a LDA topic analysis to a corpus of N documents. Due to few unlucky events all word-topic -probability distribution matrices was lost and I am left with only the document-topic probabilities $P(...
0
votes
0answers
8 views
Hopfield network dynamics with 1/0 vs +1/-1
Hopfield networks can use either 1/0 for the node activation values, or +1/-1.
Nobody ever seems to discuss what difference this makes. They just pick one and run with it. Clearly, the dynamics are ...
0
votes
0answers
12 views
Clustering data that includes a categorical variable with many different levels
I'm looking to cluster data on apartments. I have the following variables for each apartment:
Latitude
Longitude
Price
Number of bathrooms
Number of bedrooms
Amenities (washer, gym, etc.)
The ...
1
vote
1answer
27 views
Clustering data that includes a categorical variable with many different levels
I'm looking to cluster data on apartments. I have the following variables for each apartment:
Latitude
Longitude
Price
Number of bathrooms
Number of bedrooms
Amenities (washer, gym, etc.)
The ...
0
votes
0answers
25 views
Density/distribution estimation with neural networks?
I am looking for the correct approach to this problem. I have been browsing through internet and although I think I'm on the right track, I am not sure if I found the most optimal solutions. I am not ...
1
vote
0answers
10 views
Performance of Hierarchical Temporal Memory on unsupervised online anomaly detection problems
I'm looking for an algorithm to detect anomalies in streaming data (server metrics). The detection needs to be near-real time and unsupervised (labeled data will never be available, unfortunately, and ...
2
votes
0answers
12 views
What are the simple methods to do an unsupervised cluster to stock return time series?
I am a student in finance and I am working on my thesis project. I am interested in doing a clustering to stock time series. I first read the paper 'Time-series clustering – A decade review' from ...
0
votes
0answers
23 views
Remaining Skewness after log transformation
I am new in this community and I hope you could help me out.
Right now I am working on a dataset which has multiple variables that are highly skewed. Below you can see one example of a variable ...
2
votes
3answers
78 views
How to find strongly connected subgraphs in a graph? [duplicate]
I have a simple, undirected graph where I'd like to detect "natural" subgraphs where vertices are connected intensively internally but sparsely externally. The problem is that I have no exact hint ...
0
votes
0answers
16 views
What are some good probabilistic modeling approaches for co-occurance of words in large corpus of short documents?
I'd be very grateful for some advice in finding a modeling approach and toolset for a NLP problem I have.
I need to learn a joint distribution of tokens in a corpus of docs. I have on the order of ...
0
votes
0answers
74 views
Action space for Reinforcement Learning implementation
I am confused about how to set action space for my application for which I like to use Reinforcement Learning to select the best instance.
I have two groups of instances.
Group_a and Group_b, ...
0
votes
0answers
37 views
Categorical data, clustering
I have records data from library:
user-id, item-id, type of user, checkout date, gender, etc
And information about item:
author, date, publisher, publication year, title, item type, isbn, etc
I want ...
2
votes
0answers
39 views
what's the difference between semi-supervised learning and partially supervised learning? [closed]
Isn't every semi-supervised problem also a partially supervised learning problem and vice versa?
-1
votes
1answer
12 views
Which clustering technique to use for 1D data with 2 Gaussian distributions without training data?
I have 1D data which is likely to be 2 Gaussian distributions overlapped. I do not have training data but I know one should be slightly high contributed from few points and another is very broad with ...
-1
votes
1answer
19 views
K-means in R: complete case analysis followed by nearest-neighbor assignment for partial data
I have a dataset of 3K observations with only 162 being a complete case.
I have read here that it is possible to run knn on the complete cases and then conduct a nearest neighbour assignment for ...
1
vote
1answer
60 views
Is a chi-square test for independence appropriate on a contingency table where one category is the unsupervised learning cluster?
I have a data set that has been partitioned into four clusters by executing a clustering algorithm that used principal components from a principal component analysis (PCA). I then make a contingency ...
1
vote
0answers
18 views
When applying PCA to a dataset consisting of regression coefficients, should one use PCA on correlation or on covariance?
This is a follow-up question from the post: PCA on correlation or covariance?
The accepted answer quotes:
You tend to use the covariance matrix when the variable scales are
similar and the ...
0
votes
1answer
50 views
How to get meaningful results from Softmax activation in Deep Unsupervised Clustering Network
I found this interesting paper regarding deep unsupervised clustering and am looking to mimic some of the things done, however there is one thing that is not clear to me. In the paper, they use a ...
1
vote
0answers
34 views
What exactly is semisupervised learning?
I have come across two descriptions of what semisupervised learning is, where one would have a small set $\mathcal{L}$ of labeled data and a larger set $\mathcal{U}$ of unlabeled data. The first ...
0
votes
1answer
31 views
Cross Validation in application of clustering on a collection of similarly behaving time series
I'm trying to understand how and at which point can one apply Cross Validation for time series data. If i'm not wrong CV increases generalisation so that our model has less bias in case the data is ...
1
vote
2answers
37 views
Unsupervised learning with missing features
Assume I have a set of N vectors with M features each. If I want to create a latent space to project these vectors into, there are a variety of techniques available to me, such as Principle Component ...
1
vote
1answer
329 views
K-nearest neighbor supervised or unsupervised machine learning?
I've read in several papers that K-nearest neighbor can be supervised or unsupervised learning. Is Knn always unsupervised when one use it for clustering and supervised when one used it for ...
-1
votes
1answer
29 views
clustering VS supervised classification, in the case of very small database
I'm trying to classify/cluster subjects according to 4 features in two classes: healthy and sick.
Two things to know: I know the labels/classes of each subject + I only have 40 subjects (in total: ...
1
vote
0answers
23 views
Unsupervised Text Clustering Project: How to get started? [closed]
I work for a manufacturing company where robust databases and data integrity have not always been a priority. I have a very messy and finite list of 13,102 tool descriptions. I need to find out how ...
0
votes
1answer
71 views
unsupervised classification VS supervised classification when data labels are known
Can someone give me some scenario where it's better to use clustering (unsupervised classification) than supervised classification such as SVM ? I mean in a case where you know the data labels/classes....
14
votes
3answers
417 views
What *is* an Artificial Neural Network?
As we delve into Neural Networks literature, we get to identify other methods with neuromorphic topologies ("Neural-Network"-like architectures). And I'm not talking about the Universal Approximation ...
1
vote
1answer
42 views
Is it possible to select features from completely unlabeled data?
I have seen many examples of using semi-supervised learning to reduce the the number of features in a data set, but I am wondering if it is possible to somehow reduce features with purely unlabeled ...
1
vote
0answers
15 views
how many data points for effective LDA (clustering)?
Is there an advisable number of data points recommended to be able to effectively use LDA (Linear discriminant analysis)? I know this is a very open-ended question: but how many is enough?
2
votes
1answer
66 views
“Reverse” clustering?
There is a lot of content about how to cluster, say, customers (k-means, EM clustering, etc.).
However, is there a way to reverse cluster customers? Meaning, let's say I have 20 customers, and want ...
0
votes
1answer
73 views
Training and testing an autoencoder on very sparsely populated data
I am exploring the possibility of using a deep autoencoder neural net to build a recommender system. I am firstly testing the model's performance on the traditionally used benchmark of the Movielens ...
0
votes
0answers
27 views
Affinity Propagation - number of clusters?
I am working on a clustering exercise where I am unsure about how to decide what is the appropriate number of clusters for my dataset. This is an unsupervised learning exercise so I have no way to ...
0
votes
0answers
41 views
Factor Scores and Clustering
I used self organizing map to cluster factor scores. I used 5 variables for clustering and at the end, I have 4 clusters. Some factor score means are negative. How am I supposed to interpret these ...
0
votes
1answer
55 views
Difference between Hartigan & Wong Algo to Lloyd's algorithm in K-means clustering
In the iterations of Hartigan and Wong Algo of K-Means clustering, If the centroid is updated in the last step, for each data point included, the within- cluster sum of squares for each data point if ...
1
vote
0answers
254 views
Unsupervised anomaly detection - metric for tuning Isolation Forest parameters
I have a project, in which, one of the stages is to find and label anomalous data points, that are likely to be outliers.
As a first step, I am using Isolation Forest algorithm, which, after plotting ...
0
votes
0answers
54 views
agglomerative clustering of undirected weighted graph
Let's say I have a simple graph of undirected, weighted edges. I want to agglomerate nodes one-at-a-time by combining the two nodes which have the highest weighted edge, recalculating remaining edge ...
0
votes
1answer
51 views
What is the best approach of clustering if you know number of elements in each cluster?
I want to predict sporting event with knockout system tournament. Data have some features attached to each athlete. I know that there are determined number of results such as:
...
0
votes
0answers
21 views
Get rid of the disrupting points
Here on the following picture, we can identify two clouds. We can identify a lot of points that are out of the two clouds and can disrupt a bit the the training and test accuracy (%). Is there an ...
2
votes
2answers
215 views
Kmeans - Does removing outliers on a dimension affect other dimensions clustering prediction?
I have a set with several features that I wish to cluster using Kmeans. If I remove a point that is an outlier on one dimenssion but not in the others will it affect the result?
Outliers were found ...
0
votes
1answer
15 views
Should feature scaling be used while using unsupervised algorithms?
I have read many articles and resources about using feature scaling and when to use it, in particular two answers on this website as well-
When should I apply feature scaling for my data?
What ...
