All Questions
Tagged with clustering feature-selection
66 questions
0
votes
0
answers
22
views
Stock clustering based on fundamental reporting
I want to made a stock clusterization, based on their fundamental features from companies quarter reports.
I collected quarter reports from 2018 to 2022. Some companies have reports for all quarters ...
1
vote
1
answer
16
views
Google android-app's titles and soft clustering
I have a no-trial question: I want to soft cluster the apps from Google Store. Most of the parameters are numbers, so no big clue. There are also "tags" but this is like using categorical ...
- 171
1
vote
1
answer
170
views
Features differ between classes
Good evening everyone.
Regarding the topic related to Sparse Clustering (for example K-Means).
For example, in "Witten DM, Tibshirani R. A framework for feature selection in clustering" the ...
0
votes
1
answer
302
views
feature sealection/elimination and multi-collinearity in a Cox proportional hazards models?
I am looking into how differing brain tumor genetics affects patient survival. I have a gene dataset with around 4600 predictors, which are often strongly correlated with each other. Now I want to ...
- 581
1
vote
0
answers
111
views
What are pitfalls of clustering variables by absolute correlation for variable selection?
I have a 1,500,000-row dataset with about 100 binary flags and 1,100 continuous variables. Many of the continuous variables have a very high correlation with each other (...
- 428
1
vote
0
answers
30
views
Supervised for Unsupervised problem
I am currently working on clustering using R. I have formed clusters using Hclust and PAM. Now, I am trying to find the feature importance for each cluster. As I have features and cluster information, ...
- 31
0
votes
1
answer
132
views
How to select a set of independent features predictive of all other features without a target variable
Problem
I have a dataset NxD, where N - number of observation (~100k) and D - number of features (~10k)
(More specifically it is a single-cell RNAseq data, so each observation is a single cell and ...
- 35
1
vote
0
answers
26
views
Does the N-dimensional hypervolume refer to the N attributes we have in the training sets?
If we have $N$ features in the feature set that is being used to train a machine learning model, does the $N$ here cause the data points to be in hyperspace provided that $N > 3$? I'm assuming that ...
- 223
2
votes
1
answer
460
views
Does automatic feature selection for clustering helps with finding meaningful clusters?
The objective of clustering is to find interesting groups in data. My question is, whether feature selection can substantially help with this objective.
I understand feature selection can remove ...
- 129
1
vote
1
answer
357
views
Does it make sense to use feature selection methods to reduce dimensionality for unsupervised clustering?
If I have a dataset that is labeled with positive and negative examples, and I'd like to cluster (i.e. unsupervised) only the positive examples, does it make sense to reduce dimensionality using ...
- 111
1
vote
0
answers
119
views
Clustering binary data : feature selection vs Apriori
My data set is a 999 rows x 964 columns Panda DataFrame.
Each row is a user.
My data is binary : 0 for absent and 1 for present.
I would like to fit the users ...
- 11
0
votes
0
answers
489
views
finding the most correlated features in a multivariate time series
Assume I have a multivariate time series with 4 features where feature 1 is the most important one for me but at the same time I am suspecting maybe there is at least one more feature (another column ...
- 281
0
votes
1
answer
464
views
Can I use the Silhouette to measure quality of clusters in different dimensions?
Can I use the Silhouette to measure quality of clusters in different dimensions?
For example, let's say we run kmeans for some $k$ using 6 features of the dataset. Mark the resulted silhouette as $...
- 141
0
votes
1
answer
27
views
Did my text data come from two distinct distributions?
I have labeled text data from two different classes. I have calculated tfidf feature representation of all the sentences in question. I have a huge matrix where rows are sentences and columns are ...
- 299
1
vote
1
answer
189
views
Overrepresented Features in Clustering
So I was thinking, if I have a set of features (let's say $(X_1, X_2, X_3)$) that basically describe the same overarching feature $Y$, and can somehow be mapped $(X_1, X_2, X_3) \rightarrow Y$. In ...
1
vote
1
answer
85
views
Can I view the features of my clusters without doing it by hand?
I have performed hierarchical clustering on a data set with 186 participants and 94 variables for each participant.
What I want to know is if there is a way to see which features are "driving" my ...
- 11
1
vote
1
answer
109
views
Should I remove co-varying factors before clustering?
I have a data set of around 850 factors representing 150 geographical areas. I am looking to cluster these geographical areas, and I am intending to use a K-means clustering algorithm to do this. My ...
- 31
-1
votes
1
answer
94
views
How to weight features when doing text mining?
I have a case where I'm doing text mining over a list of product titles. In particular I want to run a clustering algorithm. But I also have some information about those products that I think can add ...
- 329
2
votes
2
answers
1k
views
Is it possible to select features from completely unlabeled data?
I have seen many examples of using semi-supervised learning to reduce the the number of features in a data set, but I am wondering if it is possible to somehow reduce features with purely unlabeled ...
- 205
7
votes
1
answer
271
views
Which approach based on the LASSO yields more biologically relevant results for gene data-sets?
I have a data-set with a continuous outcome variable and some confounding variables (like age, gender, ...) and many gene expressions (more than samples). The goal is to find relevant genes in ...
- 185
1
vote
1
answer
71
views
Feature Identification for customer profiling
Problem: I want to identify the characteristic(s) of people who would shop on Monday vs Sunday (or any such dichotomous response variable).
I have over a million observations and >50 variables/...
- 11
1
vote
0
answers
89
views
Dissimilarity matrices and autoencoders
As a thought experiment:
If I use an feed-forward autoencoder with sigmoidal activation to reduce a multi-dimensional dataframe of continous inputs to a two-dimensionsal representation...
Does a ...
- 31
1
vote
3
answers
2k
views
Cluster method that also returns the significance of each feature for clustering?
Is there a clustering method that returns the importance of different features for differentiating the clusters, along with the cluster assignments?
Kind of like how post-hoc analysis of decision ...
- 251
2
votes
0
answers
46
views
How to determine summary like tables on any informative web (html) page
I am struggling with determining the best way to guess which table (if any) on a given web page is the summary table. Examples would be the first, right-side tables on these pages.
http://wikitravel....
- 71
0
votes
1
answer
108
views
post-clustering feature selection high dimension data
I would like to know if there are very popular methods for feature selection following clustering with k-means or HAC.
More precisely, I used these methods on genomic data to sort a hundred patients ...
- 41
4
votes
1
answer
562
views
Before clustering binary data objects, may one remove attributes that are constant?
The question is made up and was inspired by a broader somewhat similar one.
Say there is a dataset, cases (objects) by binary (dichotomous) attributes (attribute either present ...
- 58.8k
0
votes
0
answers
35
views
How to select features for unsupervised clustering / prediction from a list of ~1000 features?
I have some very large data sets from the World Bank on economic performance and I'm trying to analyze if variable A can predict variable B, so am trying to apply
Unsupervised clustering to ...
- 121
2
votes
2
answers
3k
views
Weka clustering methods greyed out
I generated a csv file with 167 attributes and around 5000 entries. One is a nominal attribute, two are dates and the rest is numerical.
I can import the file into the weka explorer without problems. ...
- 23
0
votes
1
answer
2k
views
can the time complexity of k-nearest-neighbor allow it to be used as a clustering technique?
If I have N of about 3000 data points, each of about dimensions d of 50, and so the k in kNN is sqrt(3000/2) is about 40, then applying kNN to these points
would be about O(NdK) = O(3000*50*40) which ...
- 369
-1
votes
1
answer
35
views
Is applying Clustering and then Classification good approach to solve multi-categorical classification problems?
Like for following Data Example:
Let's assume I have following T-Shirt size categories:
Training Data set:
...
- 1
1
vote
0
answers
167
views
Evaluation of clustering algorithms
I've been working on using the Dirichlet clustering algorithm to cluster users based on their behavior.It's an unsupervised task. While the Dirichlet clustering algorithm does a good job of overcoming ...
- 111
1
vote
0
answers
93
views
Deciding Features for K means Clustering
I have a Customer Transaction Dataset belonging to an Online Retail Company with the following fields:
DateKey – The date on which the transaction occurred
CustomerKey – Customer ID of the customer ...
- 11
2
votes
1
answer
2k
views
clustering for data with too many features
I have a data set of information about different products that I want to cluster similare ones so I can do pricing on them. Each product have at least ten features that I can consider to differentiate ...
- 141
0
votes
0
answers
244
views
How do I obtain ranges of features from a Machine learning algorithm?
My problem is as follows, I will try to present it succinctly:
I have about 20k sample data with ~12 features and a label (true or false). I can easily use either a regression or a classification ...
- 139
1
vote
1
answer
46
views
Text document model with multiple zones for clustering
I have a model of a text document: Doc (content: String, title: String, date: Long, geo: array[String], persons: array[String], ...)
I need to represent this model as a vector for clustering.
How to ...
- 11
-1
votes
1
answer
265
views
Clustering data with mixed attributes: timeseries and constants?
I have a data set of time series, and each time series has a set of attributes. I know I can cluster time series using dynamic time warping and k-means (https://github.com/alexminnaar/time-series-...
- 169
5
votes
4
answers
4k
views
Can PCA allow to identify redundant variables that can be removed before doing cluster analysis?
I hope this is suitable for this forum: I am new to PCA and what I ultimately want to do is perform cluster analysis on my dataset.
I have 20 physical descriptor variables for organisms, each with 300 ...
pca_newby
1
vote
0
answers
46
views
Feature ranking for *known* clusters
I am aware of feature ranking (i.e. selecting the 'best' features) for a binary classification task based on some model, however, I was wondering how to do this in the absence of a model? For example, ...
- 111
0
votes
1
answer
81
views
Can cluster analysis of preclassified items gives idea about the classification performance?
Suppose in a classification we have a dataset with many features and their class, we want to select some features using which we can construct a classifier.
We perform the cluster evaluation for the ...
- 111
6
votes
1
answer
18k
views
How to determine which variables to be used for cluster analysis
I have about 10 variables (features) and want to do cluster analysis of cases (data points). I have a number of ideas about which variables to be included for cluster analysis:
Plot the variables ...
- 529
1
vote
0
answers
80
views
Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions
Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?:
The data points are real valued vectors in ...
- 577
0
votes
1
answer
334
views
Model-based clustering evaluation with BIC
Let's say I have fitted two models using EM-clustering and they differ in both the number of clusters and are fitted on different subset of features (chosen from the same set of features).
Could I ...
- 11
0
votes
3
answers
134
views
Suspicious results after clustering
I've done a clustering and I think that my results are too good to be trusted. Here is my pipeline:
Inputs: a dataset of 208 images, distributed into 2 classes (99 and 109 images in each class).
...
- 101
0
votes
1
answer
138
views
Measure influence of attribues on clustering
I don't have a specific example for my problem and maybe this is trivial, but I want to know how to measure the influence of specific attributes (or dimensions) of a dataset for clustering, like there ...
- 103
3
votes
1
answer
464
views
CART and Clustered Data?
Just wonder if there is any caveat if one fits regular regression trees to clustered data but ignores the clustered structure of the data. More generally, how bad it would be if we fit regression ...
- 129
15
votes
2
answers
19k
views
Feature selection for clustering problems
I am trying to make group together different datasets using unsupervised algorithms (clustering). The problem is that I have many features (~500) and a small amount of cases (200-300).
So far I used ...
- 151
0
votes
1
answer
129
views
Selecting features to find predefined groups in clustering
I have a dataset I believe is easily clustered into a few groups, however, a bunch of junk features interfere with clustering. Is there a method of eliminating these bad features within this dataset?
...
17
votes
4
answers
27k
views
Text Mining: how to cluster texts (e.g. news articles) with artificial intelligence?
I have built some neural networks (MLP (fully-connected), Elman (recurrent)) for different tasks, like playing Pong, classifying handwritten digits and stuff...
Additionally I tried to build some ...
- 2,935
0
votes
0
answers
1k
views
Feature extraction based on correlations
I have a small problem regarding feature extraction with correlation. I have divided my question in four parts hoping that somebody can help me.
I have a dataset consisting of fMRI images. Each image ...
- 1,824
0
votes
0
answers
106
views
Rescaling Features for ML
I have data that is collected every month and I want to perform K-means clustering on each month (both on historical data and on future data). However, it isn't clear to me how best to rescale my data ...
- 504